Note: This is test shard 1 of 8.
[==========] Running 8 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from AdminCliTest
[ RUN ] AdminCliTest.RebalancerReportOnly
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240426 09:48:41.887710 398 test_util.cc:268] Using random seed: -820913208
I20240426 09:48:41.895323 398 ts_itest-base.cc:112] Starting cluster with:
I20240426 09:48:41.895393 398 ts_itest-base.cc:113] --------------
I20240426 09:48:41.895411 398 ts_itest-base.cc:114] 5 tablet servers
I20240426 09:48:41.895426 398 ts_itest-base.cc:115] 3 replicas per TS
I20240426 09:48:41.895438 398 ts_itest-base.cc:116] --------------
2024-04-26T09:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-04-26T09:48:41Z Disabled control of system clock
I20240426 09:48:41.904234 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:41395
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:45347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:41395 with env {}
W20240426 09:48:42.007172 410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:42.007385 410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:42.007422 410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:42.010257 410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240426 09:48:42.010313 410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:42.010331 410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240426 09:48:42.010347 410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240426 09:48:42.013900 410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:41395
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:41395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:42.014715 410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:42.015846 410 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:42.023916 418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:42.024657 415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:42.025559 410 server_base.cc:1034] running on GCE node
W20240426 09:48:42.026199 416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:42.026684 410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:42.027581 410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:42.028730 410 hybrid_clock.cc:648] HybridClock initialized: now 1714124922028698 us; error 46 us; skew 500 ppm
I20240426 09:48:42.030593 410 webserver.cc:460] Webserver started at http://127.0.99.190:40239/ using document root <none> and password file <none>
I20240426 09:48:42.031148 410 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:42.031203 410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:42.031347 410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:42.032944 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "a6d646a42e2f434485fefdb88476da3a"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.033447 410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "a6d646a42e2f434485fefdb88476da3a"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.037186 410 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20240426 09:48:42.039186 424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.040171 410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:42.040274 410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "a6d646a42e2f434485fefdb88476da3a"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.040345 410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:42.052079 410 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:42.052662 410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:42.052806 410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:42.059111 410 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:41395
I20240426 09:48:42.059124 476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:41395 every 8 connection(s)
I20240426 09:48:42.060029 410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240426 09:48:42.062426 477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:42.067533 477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a: Bootstrap starting.
I20240426 09:48:42.069629 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 410
I20240426 09:48:42.069691 477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:42.069739 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240426 09:48:42.070338 477 log.cc:824] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:42.072021 477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a: No bootstrap required, opened a new log
I20240426 09:48:42.075618 477 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d646a42e2f434485fefdb88476da3a" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41395 } }
I20240426 09:48:42.075855 477 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:42.075917 477 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6d646a42e2f434485fefdb88476da3a, State: Initialized, Role: FOLLOWER
I20240426 09:48:42.076373 477 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d646a42e2f434485fefdb88476da3a" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41395 } }
I20240426 09:48:42.076532 477 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:42.076598 477 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:42.076706 477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:42.077625 477 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d646a42e2f434485fefdb88476da3a" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41395 } }
I20240426 09:48:42.077975 477 leader_election.cc:302] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6d646a42e2f434485fefdb88476da3a; no voters:
I20240426 09:48:42.078321 477 leader_election.cc:288] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:42.078697 482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:42.078967 482 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [term 1 LEADER]: Becoming Leader. State: Replica: a6d646a42e2f434485fefdb88476da3a, State: Running, Role: LEADER
I20240426 09:48:42.079277 482 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d646a42e2f434485fefdb88476da3a" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41395 } }
I20240426 09:48:42.079512 477 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [sys.catalog]: configured and running, proceeding with master startup.
I20240426 09:48:42.080742 483 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6d646a42e2f434485fefdb88476da3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d646a42e2f434485fefdb88476da3a" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41395 } } }
I20240426 09:48:42.080876 483 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [sys.catalog]: This master's current role is: LEADER
I20240426 09:48:42.081169 484 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6d646a42e2f434485fefdb88476da3a. Latest consensus state: current_term: 1 leader_uuid: "a6d646a42e2f434485fefdb88476da3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6d646a42e2f434485fefdb88476da3a" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 41395 } } }
I20240426 09:48:42.081280 491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 09:48:42.081332 484 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a [sys.catalog]: This master's current role is: LEADER
I20240426 09:48:42.084407 491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 09:48:42.089286 491 catalog_manager.cc:1347] Generated new cluster ID: b6d7992a57d447c8bce2b7d58cb4cf72
I20240426 09:48:42.089349 491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 09:48:42.121305 491 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 09:48:42.122220 491 catalog_manager.cc:1504] Loading token signing keys...
I20240426 09:48:42.129673 491 catalog_manager.cc:5857] T 00000000000000000000000000000000 P a6d646a42e2f434485fefdb88476da3a: Generated new TSK 0
I20240426 09:48:42.130355 491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 09:48:42.140725 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--builtin_ntp_servers=127.0.99.148:45347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20240426 09:48:42.257273 501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:42.257488 501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:42.257567 501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:42.260612 501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:42.260749 501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240426 09:48:42.264338 501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--never_fsync=true
--heap_profile_path=/tmp/kudu.501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:42.265210 501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:42.266311 501 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:42.275079 507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:42.275750 506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:42.276441 509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:42.277361 501 server_base.cc:1034] running on GCE node
I20240426 09:48:42.277711 501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:42.278234 501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:42.279376 501 hybrid_clock.cc:648] HybridClock initialized: now 1714124922279363 us; error 27 us; skew 500 ppm
I20240426 09:48:42.281437 501 webserver.cc:460] Webserver started at http://127.0.99.129:45719/ using document root <none> and password file <none>
I20240426 09:48:42.281960 501 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:42.282030 501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:42.282217 501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:42.283885 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "c39f6ff11bbe40e68bc0da8dab1383ec"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.284343 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "c39f6ff11bbe40e68bc0da8dab1383ec"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.287601 501 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240426 09:48:42.289657 515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.290635 501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240426 09:48:42.290738 501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "c39f6ff11bbe40e68bc0da8dab1383ec"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.290855 501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:42.313535 501 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:42.314183 501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:42.314339 501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:42.314894 501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:42.315775 501 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:42.315824 501 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.315882 501 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:42.315907 501 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.323797 501 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:46433
I20240426 09:48:42.323837 628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:46433 every 8 connection(s)
I20240426 09:48:42.324699 501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240426 09:48:42.327970 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 501
I20240426 09:48:42.328063 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240426 09:48:42.330335 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--builtin_ntp_servers=127.0.99.148:45347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240426 09:48:42.333745 629 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41395
I20240426 09:48:42.334009 629 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:42.334517 629 heartbeater.cc:511] Master 127.0.99.190:41395 requested a full tablet report, sending...
I20240426 09:48:42.335920 441 ts_manager.cc:194] Registered new tserver with Master: c39f6ff11bbe40e68bc0da8dab1383ec (127.0.99.129:46433)
I20240426 09:48:42.337060 441 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:39065
W20240426 09:48:42.437078 632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:42.437289 632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:42.437367 632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:42.440351 632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:42.440487 632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240426 09:48:42.444089 632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--never_fsync=true
--heap_profile_path=/tmp/kudu.632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:42.444990 632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:42.446098 632 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:42.455204 637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:42.455945 640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:42.456761 638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:42.457556 632 server_base.cc:1034] running on GCE node
I20240426 09:48:42.457937 632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:42.458488 632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:42.459630 632 hybrid_clock.cc:648] HybridClock initialized: now 1714124922459618 us; error 26 us; skew 500 ppm
I20240426 09:48:42.461480 632 webserver.cc:460] Webserver started at http://127.0.99.130:42803/ using document root <none> and password file <none>
I20240426 09:48:42.462013 632 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:42.462071 632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:42.462257 632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:42.463882 632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "cdd5765c2aaa4d039087537d4aa7b451"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.464371 632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "cdd5765c2aaa4d039087537d4aa7b451"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.467535 632 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240426 09:48:42.469511 646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.470551 632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:42.470654 632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "cdd5765c2aaa4d039087537d4aa7b451"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.470746 632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:42.491101 632 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:42.491727 632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:42.491876 632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:42.492408 632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:42.493286 632 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:42.493332 632 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.493381 632 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:42.493412 632 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.502563 632 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:43839
I20240426 09:48:42.502588 759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:43839 every 8 connection(s)
I20240426 09:48:42.503492 632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240426 09:48:42.506467 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 632
I20240426 09:48:42.506589 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240426 09:48:42.508877 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--builtin_ntp_servers=127.0.99.148:45347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240426 09:48:42.514003 760 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41395
I20240426 09:48:42.514251 760 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:42.514822 760 heartbeater.cc:511] Master 127.0.99.190:41395 requested a full tablet report, sending...
I20240426 09:48:42.515920 441 ts_manager.cc:194] Registered new tserver with Master: cdd5765c2aaa4d039087537d4aa7b451 (127.0.99.130:43839)
I20240426 09:48:42.516762 441 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:50297
W20240426 09:48:42.619529 763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:42.619729 763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:42.619769 763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:42.622960 763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:42.623064 763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240426 09:48:42.626600 763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--never_fsync=true
--heap_profile_path=/tmp/kudu.763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:42.627450 763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:42.628495 763 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:42.637090 768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:42.637782 771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:42.638526 769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:42.639264 763 server_base.cc:1034] running on GCE node
I20240426 09:48:42.639614 763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:42.640133 763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:42.641285 763 hybrid_clock.cc:648] HybridClock initialized: now 1714124922641275 us; error 34 us; skew 500 ppm
I20240426 09:48:42.643208 763 webserver.cc:460] Webserver started at http://127.0.99.131:42975/ using document root <none> and password file <none>
I20240426 09:48:42.643745 763 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:42.643806 763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:42.644011 763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:42.645686 763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.646168 763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.649490 763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240426 09:48:42.651624 777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.652601 763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:42.652707 763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.652784 763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:42.685700 763 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:42.686354 763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:42.686527 763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:42.687081 763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:42.688009 763 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:42.688056 763 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.688117 763 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:42.688151 763 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.697546 763 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:38085
I20240426 09:48:42.697569 890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:38085 every 8 connection(s)
I20240426 09:48:42.698455 763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240426 09:48:42.705652 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 763
I20240426 09:48:42.705760 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240426 09:48:42.707345 891 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41395
I20240426 09:48:42.707546 891 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:42.707818 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--builtin_ntp_servers=127.0.99.148:45347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240426 09:48:42.708091 891 heartbeater.cc:511] Master 127.0.99.190:41395 requested a full tablet report, sending...
I20240426 09:48:42.709007 441 ts_manager.cc:194] Registered new tserver with Master: 7ec4a65ee7ea4a23b8c73ccfdbb6d294 (127.0.99.131:38085)
I20240426 09:48:42.709484 441 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:32857
W20240426 09:48:42.816126 894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:42.816358 894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:42.816433 894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:42.819353 894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:42.819489 894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240426 09:48:42.823061 894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--never_fsync=true
--heap_profile_path=/tmp/kudu.894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:42.823979 894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:42.825116 894 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:42.833717 899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:42.834400 900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:42.835107 902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:42.835892 894 server_base.cc:1034] running on GCE node
I20240426 09:48:42.836460 894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:42.837035 894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:42.838199 894 hybrid_clock.cc:648] HybridClock initialized: now 1714124922838183 us; error 30 us; skew 500 ppm
I20240426 09:48:42.840194 894 webserver.cc:460] Webserver started at http://127.0.99.132:38787/ using document root <none> and password file <none>
I20240426 09:48:42.840739 894 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:42.840798 894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:42.840970 894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:42.842586 894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "1b5c2165a6214ec3b25d3333028b96d7"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.843057 894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "1b5c2165a6214ec3b25d3333028b96d7"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.846240 894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20240426 09:48:42.848304 908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.849237 894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240426 09:48:42.849334 894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "1b5c2165a6214ec3b25d3333028b96d7"
format_stamp: "Formatted at 2024-04-26 09:48:42 on dist-test-slave-1zqj"
I20240426 09:48:42.849408 894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:42.865061 894 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:42.865648 894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:42.865800 894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:42.866326 894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:42.867229 894 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:42.867274 894 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.867331 894 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:42.867359 894 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:42.876210 894 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:38927
I20240426 09:48:42.876227 1021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:38927 every 8 connection(s)
I20240426 09:48:42.877117 894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240426 09:48:42.884245 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 894
I20240426 09:48:42.884370 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240426 09:48:42.886137 1022 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41395
I20240426 09:48:42.886358 1022 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:42.886507 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:0
--local_ip_for_outbound_sockets=127.0.99.133
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--builtin_ntp_servers=127.0.99.148:45347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240426 09:48:42.886955 1022 heartbeater.cc:511] Master 127.0.99.190:41395 requested a full tablet report, sending...
I20240426 09:48:42.887871 440 ts_manager.cc:194] Registered new tserver with Master: 1b5c2165a6214ec3b25d3333028b96d7 (127.0.99.132:38927)
I20240426 09:48:42.888417 440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:59233
W20240426 09:48:42.992743 1025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:42.992955 1025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:42.993031 1025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:42.996073 1025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:42.996202 1025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240426 09:48:42.999588 1025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:45347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:41395
--never_fsync=true
--heap_profile_path=/tmp/kudu.1025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:43.000414 1025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:43.001505 1025 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:43.010107 1030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:43.010730 1031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:43.011490 1033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:43.012286 1025 server_base.cc:1034] running on GCE node
I20240426 09:48:43.012655 1025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:43.013216 1025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:43.014357 1025 hybrid_clock.cc:648] HybridClock initialized: now 1714124923014341 us; error 28 us; skew 500 ppm
I20240426 09:48:43.016232 1025 webserver.cc:460] Webserver started at http://127.0.99.133:39073/ using document root <none> and password file <none>
I20240426 09:48:43.016763 1025 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:43.016819 1025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:43.016986 1025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:43.018545 1025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "abce4f491e7f46b09a33a2cb7fdde1f2"
format_stamp: "Formatted at 2024-04-26 09:48:43 on dist-test-slave-1zqj"
I20240426 09:48:43.019040 1025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "abce4f491e7f46b09a33a2cb7fdde1f2"
format_stamp: "Formatted at 2024-04-26 09:48:43 on dist-test-slave-1zqj"
I20240426 09:48:43.022303 1025 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240426 09:48:43.024582 1039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:43.025527 1025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:43.025638 1025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "abce4f491e7f46b09a33a2cb7fdde1f2"
format_stamp: "Formatted at 2024-04-26 09:48:43 on dist-test-slave-1zqj"
I20240426 09:48:43.025725 1025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:43.038254 1025 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:43.038910 1025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:43.039070 1025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:43.039580 1025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:43.040509 1025 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:43.040560 1025 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:43.040621 1025 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:43.040650 1025 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:43.049938 1152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:34735 every 8 connection(s)
I20240426 09:48:43.049919 1025 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:34735
I20240426 09:48:43.050921 1025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240426 09:48:43.051952 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 1025
I20240426 09:48:43.052062 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240426 09:48:43.060245 1153 heartbeater.cc:347] Connected to a master server at 127.0.99.190:41395
I20240426 09:48:43.060465 1153 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:43.060976 1153 heartbeater.cc:511] Master 127.0.99.190:41395 requested a full tablet report, sending...
I20240426 09:48:43.061861 440 ts_manager.cc:194] Registered new tserver with Master: abce4f491e7f46b09a33a2cb7fdde1f2 (127.0.99.133:34735)
I20240426 09:48:43.062428 440 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:51017
I20240426 09:48:43.066242 398 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240426 09:48:43.079617 440 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52280:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240426 09:48:43.097407 694 tablet_service.cc:1467] Processing CreateTablet for tablet 654acc60a4e247c4bff8752cb066ceeb (DEFAULT_TABLE table=TestTable [id=5073b5481e3c471594341f72353229cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:43.097407 825 tablet_service.cc:1467] Processing CreateTablet for tablet 654acc60a4e247c4bff8752cb066ceeb (DEFAULT_TABLE table=TestTable [id=5073b5481e3c471594341f72353229cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:43.098424 694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 654acc60a4e247c4bff8752cb066ceeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:43.098423 825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 654acc60a4e247c4bff8752cb066ceeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:43.103610 956 tablet_service.cc:1467] Processing CreateTablet for tablet 654acc60a4e247c4bff8752cb066ceeb (DEFAULT_TABLE table=TestTable [id=5073b5481e3c471594341f72353229cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:43.104398 956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 654acc60a4e247c4bff8752cb066ceeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:43.104830 1171 tablet_bootstrap.cc:492] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294: Bootstrap starting.
I20240426 09:48:43.107069 1170 tablet_bootstrap.cc:492] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451: Bootstrap starting.
I20240426 09:48:43.107121 1171 tablet_bootstrap.cc:654] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:43.107939 1171 log.cc:824] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:43.108806 1170 tablet_bootstrap.cc:654] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:43.109411 1170 log.cc:824] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:43.109743 1171 tablet_bootstrap.cc:492] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294: No bootstrap required, opened a new log
I20240426 09:48:43.109956 1171 ts_tablet_manager.cc:1367] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240426 09:48:43.110836 1174 tablet_bootstrap.cc:492] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7: Bootstrap starting.
I20240426 09:48:43.110962 1170 tablet_bootstrap.cc:492] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451: No bootstrap required, opened a new log
I20240426 09:48:43.111121 1170 ts_tablet_manager.cc:1367] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:48:43.112931 1174 tablet_bootstrap.cc:654] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:43.113112 1171 raft_consensus.cc:348] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } }
I20240426 09:48:43.113291 1171 raft_consensus.cc:374] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:43.113364 1171 raft_consensus.cc:729] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4a65ee7ea4a23b8c73ccfdbb6d294, State: Initialized, Role: FOLLOWER
I20240426 09:48:43.113677 1174 log.cc:824] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:43.113777 1171 consensus_queue.cc:260] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } }
I20240426 09:48:43.114279 1170 raft_consensus.cc:348] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } }
I20240426 09:48:43.114447 891 heartbeater.cc:503] Master 127.0.99.190:41395 was elected leader, sending a full tablet report...
I20240426 09:48:43.114665 1171 ts_tablet_manager.cc:1398] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:48:43.114698 1170 raft_consensus.cc:374] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:43.114753 1170 raft_consensus.cc:729] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd5765c2aaa4d039087537d4aa7b451, State: Initialized, Role: FOLLOWER
I20240426 09:48:43.115638 1170 consensus_queue.cc:260] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } }
I20240426 09:48:43.115821 1174 tablet_bootstrap.cc:492] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7: No bootstrap required, opened a new log
I20240426 09:48:43.115981 1174 ts_tablet_manager.cc:1367] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240426 09:48:43.116397 1170 ts_tablet_manager.cc:1398] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20240426 09:48:43.116756 760 heartbeater.cc:503] Master 127.0.99.190:41395 was elected leader, sending a full tablet report...
I20240426 09:48:43.119942 1174 raft_consensus.cc:348] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } }
I20240426 09:48:43.120133 1174 raft_consensus.cc:374] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:43.120244 1174 raft_consensus.cc:729] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b5c2165a6214ec3b25d3333028b96d7, State: Initialized, Role: FOLLOWER
I20240426 09:48:43.120606 1174 consensus_queue.cc:260] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } }
I20240426 09:48:43.121215 1022 heartbeater.cc:503] Master 127.0.99.190:41395 was elected leader, sending a full tablet report...
I20240426 09:48:43.121361 1174 ts_tablet_manager.cc:1398] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20240426 09:48:43.127879 1023 tablet.cc:2369] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240426 09:48:43.199718 892 tablet.cc:2369] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:48:43.247231 1175 raft_consensus.cc:482] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:48:43.247427 1175 raft_consensus.cc:504] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } }
I20240426 09:48:43.248505 1175 leader_election.cc:288] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdd5765c2aaa4d039087537d4aa7b451 (127.0.99.130:43839), 1b5c2165a6214ec3b25d3333028b96d7 (127.0.99.132:38927)
I20240426 09:48:43.252070 976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654acc60a4e247c4bff8752cb066ceeb" candidate_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b5c2165a6214ec3b25d3333028b96d7" is_pre_election: true
I20240426 09:48:43.252075 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654acc60a4e247c4bff8752cb066ceeb" candidate_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd5765c2aaa4d039087537d4aa7b451" is_pre_election: true
I20240426 09:48:43.252379 976 raft_consensus.cc:2454] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec4a65ee7ea4a23b8c73ccfdbb6d294 in term 0.
I20240426 09:48:43.252385 714 raft_consensus.cc:2454] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec4a65ee7ea4a23b8c73ccfdbb6d294 in term 0.
I20240426 09:48:43.252787 781 leader_election.cc:302] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4a65ee7ea4a23b8c73ccfdbb6d294, cdd5765c2aaa4d039087537d4aa7b451; no voters:
I20240426 09:48:43.253023 1175 raft_consensus.cc:2781] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:48:43.253090 1175 raft_consensus.cc:482] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:48:43.253122 1175 raft_consensus.cc:3037] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:43.254004 1175 raft_consensus.cc:504] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } }
I20240426 09:48:43.254330 1175 leader_election.cc:288] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [CANDIDATE]: Term 1 election: Requested vote from peers cdd5765c2aaa4d039087537d4aa7b451 (127.0.99.130:43839), 1b5c2165a6214ec3b25d3333028b96d7 (127.0.99.132:38927)
I20240426 09:48:43.254642 976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654acc60a4e247c4bff8752cb066ceeb" candidate_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b5c2165a6214ec3b25d3333028b96d7"
I20240426 09:48:43.254673 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654acc60a4e247c4bff8752cb066ceeb" candidate_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd5765c2aaa4d039087537d4aa7b451"
I20240426 09:48:43.254747 976 raft_consensus.cc:3037] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:43.254810 714 raft_consensus.cc:3037] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451 [term 0 FOLLOWER]: Advancing to term 1
W20240426 09:48:43.254822 761 tablet.cc:2369] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:48:43.255611 976 raft_consensus.cc:2454] T 654acc60a4e247c4bff8752cb066ceeb P 1b5c2165a6214ec3b25d3333028b96d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4a65ee7ea4a23b8c73ccfdbb6d294 in term 1.
I20240426 09:48:43.255873 714 raft_consensus.cc:2454] T 654acc60a4e247c4bff8752cb066ceeb P cdd5765c2aaa4d039087537d4aa7b451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4a65ee7ea4a23b8c73ccfdbb6d294 in term 1.
I20240426 09:48:43.255887 780 leader_election.cc:302] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b5c2165a6214ec3b25d3333028b96d7, 7ec4a65ee7ea4a23b8c73ccfdbb6d294; no voters:
I20240426 09:48:43.256034 1175 raft_consensus.cc:2781] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:43.256261 1175 raft_consensus.cc:686] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec4a65ee7ea4a23b8c73ccfdbb6d294, State: Running, Role: LEADER
I20240426 09:48:43.256572 1175 consensus_queue.cc:237] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } }
I20240426 09:48:43.259399 440 catalog_manager.cc:5484] T 654acc60a4e247c4bff8752cb066ceeb P 7ec4a65ee7ea4a23b8c73ccfdbb6d294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec4a65ee7ea4a23b8c73ccfdbb6d294 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4a65ee7ea4a23b8c73ccfdbb6d294" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdd5765c2aaa4d039087537d4aa7b451" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b5c2165a6214ec3b25d3333028b96d7" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38927 } health_report { overall_health: UNKNOWN } } }
I20240426 09:48:43.284099 398 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240426 09:48:43.285624 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver c39f6ff11bbe40e68bc0da8dab1383ec to finish bootstrapping
I20240426 09:48:43.291236 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver cdd5765c2aaa4d039087537d4aa7b451 to finish bootstrapping
I20240426 09:48:43.295823 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 7ec4a65ee7ea4a23b8c73ccfdbb6d294 to finish bootstrapping
I20240426 09:48:43.300269 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 1b5c2165a6214ec3b25d3333028b96d7 to finish bootstrapping
I20240426 09:48:43.304592 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver abce4f491e7f46b09a33a2cb7fdde1f2 to finish bootstrapping
I20240426 09:48:43.339536 629 heartbeater.cc:503] Master 127.0.99.190:41395 was elected leader, sending a full tablet report...
I20240426 09:48:43.496310 956 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:43.496755 825 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:43.498368 563 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:43.499673 694 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:43.500192 1087 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:43.591724 563 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:43.593173 825 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:43.593958 1087 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:43.595446 956 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:43.603382 694 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:43.633862 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 501
I20240426 09:48:43.642000 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 632
I20240426 09:48:43.648031 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 763
I20240426 09:48:43.653857 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 894
I20240426 09:48:43.659152 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 1025
I20240426 09:48:43.664420 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 410
2024-04-26T09:48:43Z chronyd exiting
[ OK ] AdminCliTest.RebalancerReportOnly (1800 ms)
[----------] 1 test from AdminCliTest (1800 ms total)
[----------] 1 test from RebalanceStartCriteriaTest
[ RUN ] RebalanceStartCriteriaTest.TabletServerIsDown/0
I20240426 09:48:43.686375 398 test_util.cc:268] Using random seed: -819114526
I20240426 09:48:43.687129 398 ts_itest-base.cc:112] Starting cluster with:
I20240426 09:48:43.687165 398 ts_itest-base.cc:113] --------------
I20240426 09:48:43.687176 398 ts_itest-base.cc:114] 5 tablet servers
I20240426 09:48:43.687186 398 ts_itest-base.cc:115] 3 replicas per TS
I20240426 09:48:43.687196 398 ts_itest-base.cc:116] --------------
2024-04-26T09:48:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-04-26T09:48:43Z Disabled control of system clock
I20240426 09:48:43.695041 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:34561
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:38631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:34561
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240426 09:48:43.800478 1260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:43.800700 1260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:43.800760 1260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:43.803709 1260 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:43.803784 1260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240426 09:48:43.803804 1260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:43.803822 1260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240426 09:48:43.803839 1260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240426 09:48:43.807322 1260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:34561
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:34561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.1260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:43.808199 1260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:43.809290 1260 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:43.817101 1268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:43.817786 1266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:43.818569 1265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:43.819406 1260 server_base.cc:1034] running on GCE node
I20240426 09:48:43.819862 1260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:43.820742 1260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:43.821883 1260 hybrid_clock.cc:648] HybridClock initialized: now 1714124923821871 us; error 25 us; skew 500 ppm
I20240426 09:48:43.828650 1260 webserver.cc:460] Webserver started at http://127.0.99.190:33725/ using document root <none> and password file <none>
I20240426 09:48:43.829207 1260 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:43.829267 1260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:43.829453 1260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:43.831075 1260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "3e0e166e7a244b499ac9fb566f54bcdb"
format_stamp: "Formatted at 2024-04-26 09:48:43 on dist-test-slave-1zqj"
I20240426 09:48:43.831511 1260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "3e0e166e7a244b499ac9fb566f54bcdb"
format_stamp: "Formatted at 2024-04-26 09:48:43 on dist-test-slave-1zqj"
I20240426 09:48:43.834821 1260 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240426 09:48:43.836966 1274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:43.837944 1260 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:43.838048 1260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "3e0e166e7a244b499ac9fb566f54bcdb"
format_stamp: "Formatted at 2024-04-26 09:48:43 on dist-test-slave-1zqj"
I20240426 09:48:43.838153 1260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:43.855485 1260 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:43.856145 1260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:43.856312 1260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:43.862663 1260 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:34561
I20240426 09:48:43.862674 1326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:34561 every 8 connection(s)
I20240426 09:48:43.863667 1260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240426 09:48:43.866216 1327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:43.870152 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 1260
I20240426 09:48:43.870255 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240426 09:48:43.871325 1327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb: Bootstrap starting.
I20240426 09:48:43.873400 1327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:43.874190 1327 log.cc:824] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:43.876009 1327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb: No bootstrap required, opened a new log
I20240426 09:48:43.878527 1327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0e166e7a244b499ac9fb566f54bcdb" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 34561 } }
I20240426 09:48:43.878705 1327 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:43.878795 1327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e0e166e7a244b499ac9fb566f54bcdb, State: Initialized, Role: FOLLOWER
I20240426 09:48:43.879181 1327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0e166e7a244b499ac9fb566f54bcdb" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 34561 } }
I20240426 09:48:43.879297 1327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:43.879357 1327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:43.879454 1327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:43.880151 1327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0e166e7a244b499ac9fb566f54bcdb" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 34561 } }
I20240426 09:48:43.880437 1327 leader_election.cc:302] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e0e166e7a244b499ac9fb566f54bcdb; no voters:
I20240426 09:48:43.880689 1327 leader_election.cc:288] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:43.880774 1332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:43.880932 1332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [term 1 LEADER]: Becoming Leader. State: Replica: 3e0e166e7a244b499ac9fb566f54bcdb, State: Running, Role: LEADER
I20240426 09:48:43.881193 1332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0e166e7a244b499ac9fb566f54bcdb" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 34561 } }
I20240426 09:48:43.881758 1327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [sys.catalog]: configured and running, proceeding with master startup.
I20240426 09:48:43.882975 1334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e0e166e7a244b499ac9fb566f54bcdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0e166e7a244b499ac9fb566f54bcdb" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 34561 } } }
I20240426 09:48:43.883091 1334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [sys.catalog]: This master's current role is: LEADER
I20240426 09:48:43.883370 1333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e0e166e7a244b499ac9fb566f54bcdb. Latest consensus state: current_term: 1 leader_uuid: "3e0e166e7a244b499ac9fb566f54bcdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0e166e7a244b499ac9fb566f54bcdb" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 34561 } } }
I20240426 09:48:43.883466 1333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb [sys.catalog]: This master's current role is: LEADER
I20240426 09:48:43.883875 1342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 09:48:43.886251 1342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 09:48:43.890830 1342 catalog_manager.cc:1347] Generated new cluster ID: 9cd07223ccff4d4b88d967ca655d78ac
I20240426 09:48:43.890897 1342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 09:48:43.924881 1342 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 09:48:43.926190 1342 catalog_manager.cc:1504] Loading token signing keys...
I20240426 09:48:43.938529 1342 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 3e0e166e7a244b499ac9fb566f54bcdb: Generated new TSK 0
I20240426 09:48:43.939435 1342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 09:48:43.950961 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--builtin_ntp_servers=127.0.99.148:38631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:48:44.055902 1351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:44.056125 1351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:44.056200 1351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:44.059114 1351 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:44.059183 1351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:44.059278 1351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240426 09:48:44.062713 1351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--never_fsync=true
--heap_profile_path=/tmp/kudu.1351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:44.063601 1351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:44.064689 1351 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:44.073304 1357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.074018 1356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.074807 1359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:44.075724 1351 server_base.cc:1034] running on GCE node
I20240426 09:48:44.076120 1351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:44.076704 1351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:44.077847 1351 hybrid_clock.cc:648] HybridClock initialized: now 1714124924077831 us; error 33 us; skew 500 ppm
I20240426 09:48:44.079849 1351 webserver.cc:460] Webserver started at http://127.0.99.129:43909/ using document root <none> and password file <none>
I20240426 09:48:44.080405 1351 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:44.080462 1351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:44.080647 1351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:44.082352 1351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "139bff50b7a24569b04063fefa9c725f"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.082845 1351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "139bff50b7a24569b04063fefa9c725f"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.086092 1351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240426 09:48:44.088213 1365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.089156 1351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:48:44.089264 1351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "139bff50b7a24569b04063fefa9c725f"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.089347 1351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:44.108592 1351 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:44.109249 1351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:44.109408 1351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:44.109928 1351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:44.110813 1351 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:44.110869 1351 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.110913 1351 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:44.110942 1351 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.119014 1351 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:45429
I20240426 09:48:44.119040 1478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:45429 every 8 connection(s)
I20240426 09:48:44.119931 1351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240426 09:48:44.126530 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 1351
I20240426 09:48:44.126636 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240426 09:48:44.128150 1479 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34561
I20240426 09:48:44.128391 1479 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:44.128846 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--builtin_ntp_servers=127.0.99.148:38631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:44.128921 1479 heartbeater.cc:511] Master 127.0.99.190:34561 requested a full tablet report, sending...
I20240426 09:48:44.130257 1291 ts_manager.cc:194] Registered new tserver with Master: 139bff50b7a24569b04063fefa9c725f (127.0.99.129:45429)
I20240426 09:48:44.131394 1291 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:47281
W20240426 09:48:44.234954 1482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:44.235173 1482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:44.235219 1482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:44.238322 1482 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:44.238394 1482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:44.238476 1482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240426 09:48:44.242077 1482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--never_fsync=true
--heap_profile_path=/tmp/kudu.1482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:44.243028 1482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:44.244164 1482 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:44.252663 1488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.253533 1487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.254304 1490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:44.255064 1482 server_base.cc:1034] running on GCE node
I20240426 09:48:44.255406 1482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:44.255900 1482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:44.257042 1482 hybrid_clock.cc:648] HybridClock initialized: now 1714124924257035 us; error 23 us; skew 500 ppm
I20240426 09:48:44.258819 1482 webserver.cc:460] Webserver started at http://127.0.99.130:43355/ using document root <none> and password file <none>
I20240426 09:48:44.259326 1482 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:44.259374 1482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:44.259510 1482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:44.261046 1482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "83d022dc1b4e40af83870a08512c582a"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.261468 1482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "83d022dc1b4e40af83870a08512c582a"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.264505 1482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240426 09:48:44.266458 1496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.267416 1482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:44.267503 1482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "83d022dc1b4e40af83870a08512c582a"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.267571 1482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:44.286729 1482 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:44.287375 1482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:44.287530 1482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:44.288030 1482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:44.288872 1482 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:44.288923 1482 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.288969 1482 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:44.288987 1482 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.297134 1482 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:44399
I20240426 09:48:44.297179 1609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:44399 every 8 connection(s)
I20240426 09:48:44.298022 1482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240426 09:48:44.304409 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 1482
I20240426 09:48:44.304513 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240426 09:48:44.306509 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--builtin_ntp_servers=127.0.99.148:38631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:44.306538 1610 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34561
I20240426 09:48:44.306748 1610 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:44.307312 1610 heartbeater.cc:511] Master 127.0.99.190:34561 requested a full tablet report, sending...
I20240426 09:48:44.308267 1291 ts_manager.cc:194] Registered new tserver with Master: 83d022dc1b4e40af83870a08512c582a (127.0.99.130:44399)
I20240426 09:48:44.308769 1291 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:53551
W20240426 09:48:44.411996 1613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:44.412200 1613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:44.412295 1613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:44.415266 1613 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:44.415328 1613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:44.415421 1613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240426 09:48:44.418879 1613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--never_fsync=true
--heap_profile_path=/tmp/kudu.1613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:44.419713 1613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:44.420751 1613 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:44.429273 1621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.429960 1619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.430639 1618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:44.431488 1613 server_base.cc:1034] running on GCE node
I20240426 09:48:44.431876 1613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:44.432476 1613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:44.433633 1613 hybrid_clock.cc:648] HybridClock initialized: now 1714124924433621 us; error 32 us; skew 500 ppm
I20240426 09:48:44.435596 1613 webserver.cc:460] Webserver started at http://127.0.99.131:46471/ using document root <none> and password file <none>
I20240426 09:48:44.436161 1613 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:44.436223 1613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:44.436408 1613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:44.438102 1613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "3371a27570a94a02bcd4109e88a9fc6d"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.438563 1613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "3371a27570a94a02bcd4109e88a9fc6d"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.441890 1613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240426 09:48:44.443979 1627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.444901 1613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:48:44.445001 1613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "3371a27570a94a02bcd4109e88a9fc6d"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.445075 1613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:44.461344 1613 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:44.461944 1613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:44.462095 1613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:44.462590 1613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:44.463485 1613 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:44.463531 1613 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.463586 1613 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:44.463613 1613 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.471364 1613 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:38183
I20240426 09:48:44.471419 1740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:38183 every 8 connection(s)
I20240426 09:48:44.472313 1613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240426 09:48:44.472849 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 1613
I20240426 09:48:44.472942 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240426 09:48:44.475106 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--builtin_ntp_servers=127.0.99.148:38631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:44.481034 1741 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34561
I20240426 09:48:44.481240 1741 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:44.481649 1741 heartbeater.cc:511] Master 127.0.99.190:34561 requested a full tablet report, sending...
I20240426 09:48:44.482566 1291 ts_manager.cc:194] Registered new tserver with Master: 3371a27570a94a02bcd4109e88a9fc6d (127.0.99.131:38183)
I20240426 09:48:44.483153 1291 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:45327
W20240426 09:48:44.579646 1744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:44.579891 1744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:44.579959 1744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:44.583048 1744 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:44.583132 1744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:44.583221 1744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240426 09:48:44.586743 1744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--never_fsync=true
--heap_profile_path=/tmp/kudu.1744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:44.587690 1744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:44.588807 1744 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:44.597241 1750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.597967 1752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.598734 1749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:44.599589 1744 server_base.cc:1034] running on GCE node
I20240426 09:48:44.599957 1744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:44.600513 1744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:44.601665 1744 hybrid_clock.cc:648] HybridClock initialized: now 1714124924601652 us; error 27 us; skew 500 ppm
I20240426 09:48:44.603557 1744 webserver.cc:460] Webserver started at http://127.0.99.132:37901/ using document root <none> and password file <none>
I20240426 09:48:44.604110 1744 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:44.604162 1744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:44.604336 1744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:44.605976 1744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "582e0061fda545bb95efae811651d5d7"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.606420 1744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "582e0061fda545bb95efae811651d5d7"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.609645 1744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240426 09:48:44.611676 1758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.612618 1744 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:48:44.612712 1744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "582e0061fda545bb95efae811651d5d7"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.612797 1744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:44.623148 1744 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:44.623685 1744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:44.623831 1744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:44.624331 1744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:44.625140 1744 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:44.625185 1744 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.625255 1744 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:44.625288 1744 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.633406 1744 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:35431
I20240426 09:48:44.633423 1871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:35431 every 8 connection(s)
I20240426 09:48:44.634310 1744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240426 09:48:44.640887 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 1744
I20240426 09:48:44.641007 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240426 09:48:44.642590 1872 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34561
I20240426 09:48:44.642838 1872 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:44.643174 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:0
--local_ip_for_outbound_sockets=127.0.99.133
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--builtin_ntp_servers=127.0.99.148:38631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:44.643270 1872 heartbeater.cc:511] Master 127.0.99.190:34561 requested a full tablet report, sending...
I20240426 09:48:44.644114 1291 ts_manager.cc:194] Registered new tserver with Master: 582e0061fda545bb95efae811651d5d7 (127.0.99.132:35431)
I20240426 09:48:44.644845 1291 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:43613
W20240426 09:48:44.746675 1875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:44.746933 1875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:44.747005 1875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:44.750065 1875 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:44.750147 1875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:44.750241 1875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240426 09:48:44.753834 1875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:38631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:34561
--never_fsync=true
--heap_profile_path=/tmp/kudu.1875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:44.754796 1875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:44.755911 1875 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:44.764663 1880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.765411 1883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:44.766140 1881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:44.766980 1875 server_base.cc:1034] running on GCE node
I20240426 09:48:44.767369 1875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:44.767921 1875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:44.769091 1875 hybrid_clock.cc:648] HybridClock initialized: now 1714124924769074 us; error 34 us; skew 500 ppm
I20240426 09:48:44.771128 1875 webserver.cc:460] Webserver started at http://127.0.99.133:45813/ using document root <none> and password file <none>
I20240426 09:48:44.771673 1875 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:44.771725 1875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:44.771888 1875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:44.773556 1875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "a2763d16993042bd966e8533f0ccba81"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.774003 1875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "a2763d16993042bd966e8533f0ccba81"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.777338 1875 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240426 09:48:44.779506 1889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.780488 1875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:44.780584 1875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "a2763d16993042bd966e8533f0ccba81"
format_stamp: "Formatted at 2024-04-26 09:48:44 on dist-test-slave-1zqj"
I20240426 09:48:44.780661 1875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:44.805253 1875 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:44.805846 1875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:44.806002 1875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:44.806519 1875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:44.807430 1875 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:44.807475 1875 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.807521 1875 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:44.807551 1875 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:44.815526 1875 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:38045
I20240426 09:48:44.815555 2002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:38045 every 8 connection(s)
I20240426 09:48:44.816457 1875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240426 09:48:44.818933 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 1875
I20240426 09:48:44.819031 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240426 09:48:44.824817 2003 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34561
I20240426 09:48:44.825026 2003 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:44.825418 2003 heartbeater.cc:511] Master 127.0.99.190:34561 requested a full tablet report, sending...
I20240426 09:48:44.826284 1291 ts_manager.cc:194] Registered new tserver with Master: a2763d16993042bd966e8533f0ccba81 (127.0.99.133:38045)
I20240426 09:48:44.826834 1291 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:39371
I20240426 09:48:44.832657 398 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240426 09:48:44.841004 1291 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42920:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240426 09:48:44.858580 1937 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc5b6623dbd4f3c9a50d2da74344189 (DEFAULT_TABLE table=TestTable [id=44ff20127f7f4dc28b99d213a6b01dec]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:44.858623 1675 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc5b6623dbd4f3c9a50d2da74344189 (DEFAULT_TABLE table=TestTable [id=44ff20127f7f4dc28b99d213a6b01dec]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:44.859549 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bc5b6623dbd4f3c9a50d2da74344189. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:44.859551 1937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bc5b6623dbd4f3c9a50d2da74344189. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:44.859318 1544 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc5b6623dbd4f3c9a50d2da74344189 (DEFAULT_TABLE table=TestTable [id=44ff20127f7f4dc28b99d213a6b01dec]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:44.860042 1544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bc5b6623dbd4f3c9a50d2da74344189. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:44.866135 2020 tablet_bootstrap.cc:492] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a: Bootstrap starting.
I20240426 09:48:44.867090 2021 tablet_bootstrap.cc:492] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d: Bootstrap starting.
I20240426 09:48:44.867950 2020 tablet_bootstrap.cc:654] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:44.868594 2020 log.cc:824] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:44.868891 2021 tablet_bootstrap.cc:654] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:44.869606 2021 log.cc:824] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:44.870107 2020 tablet_bootstrap.cc:492] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a: No bootstrap required, opened a new log
I20240426 09:48:44.870359 2022 tablet_bootstrap.cc:492] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81: Bootstrap starting.
I20240426 09:48:44.870386 2020 ts_tablet_manager.cc:1367] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:48:44.872356 2021 tablet_bootstrap.cc:492] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d: No bootstrap required, opened a new log
I20240426 09:48:44.872581 2021 ts_tablet_manager.cc:1367] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240426 09:48:44.872627 2022 tablet_bootstrap.cc:654] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:44.873330 2022 log.cc:824] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:44.873773 2020 raft_consensus.cc:348] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } }
I20240426 09:48:44.873987 2020 raft_consensus.cc:374] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:44.874038 2020 raft_consensus.cc:729] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83d022dc1b4e40af83870a08512c582a, State: Initialized, Role: FOLLOWER
I20240426 09:48:44.874501 2020 consensus_queue.cc:260] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } }
I20240426 09:48:44.874794 2022 tablet_bootstrap.cc:492] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81: No bootstrap required, opened a new log
I20240426 09:48:44.874936 2022 ts_tablet_manager.cc:1367] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:48:44.875432 2020 ts_tablet_manager.cc:1398] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:48:44.875813 1610 heartbeater.cc:503] Master 127.0.99.190:34561 was elected leader, sending a full tablet report...
I20240426 09:48:44.876924 2021 raft_consensus.cc:348] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } }
I20240426 09:48:44.877188 2021 raft_consensus.cc:374] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:44.877295 2021 raft_consensus.cc:729] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3371a27570a94a02bcd4109e88a9fc6d, State: Initialized, Role: FOLLOWER
I20240426 09:48:44.877673 2021 consensus_queue.cc:260] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } }
I20240426 09:48:44.878273 1741 heartbeater.cc:503] Master 127.0.99.190:34561 was elected leader, sending a full tablet report...
I20240426 09:48:44.878367 2021 ts_tablet_manager.cc:1398] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240426 09:48:44.880043 2022 raft_consensus.cc:348] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } }
I20240426 09:48:44.880254 2022 raft_consensus.cc:374] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:44.880312 2022 raft_consensus.cc:729] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2763d16993042bd966e8533f0ccba81, State: Initialized, Role: FOLLOWER
I20240426 09:48:44.880717 2022 consensus_queue.cc:260] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } }
I20240426 09:48:44.881451 2003 heartbeater.cc:503] Master 127.0.99.190:34561 was elected leader, sending a full tablet report...
I20240426 09:48:44.881688 2022 ts_tablet_manager.cc:1398] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20240426 09:48:44.973313 1742 tablet.cc:2369] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240426 09:48:45.049203 1611 tablet.cc:2369] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:48:45.059438 2027 raft_consensus.cc:482] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:48:45.059667 2027 raft_consensus.cc:504] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } }
I20240426 09:48:45.060873 2027 leader_election.cc:288] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2763d16993042bd966e8533f0ccba81 (127.0.99.133:38045), 83d022dc1b4e40af83870a08512c582a (127.0.99.130:44399)
I20240426 09:48:45.064612 1564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc5b6623dbd4f3c9a50d2da74344189" candidate_uuid: "3371a27570a94a02bcd4109e88a9fc6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83d022dc1b4e40af83870a08512c582a" is_pre_election: true
I20240426 09:48:45.064620 1957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc5b6623dbd4f3c9a50d2da74344189" candidate_uuid: "3371a27570a94a02bcd4109e88a9fc6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2763d16993042bd966e8533f0ccba81" is_pre_election: true
I20240426 09:48:45.064889 1957 raft_consensus.cc:2454] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3371a27570a94a02bcd4109e88a9fc6d in term 0.
I20240426 09:48:45.064900 1564 raft_consensus.cc:2454] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3371a27570a94a02bcd4109e88a9fc6d in term 0.
I20240426 09:48:45.065344 1630 leader_election.cc:302] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3371a27570a94a02bcd4109e88a9fc6d, 83d022dc1b4e40af83870a08512c582a; no voters:
I20240426 09:48:45.065621 2027 raft_consensus.cc:2781] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:48:45.065691 2027 raft_consensus.cc:482] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:48:45.065721 2027 raft_consensus.cc:3037] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:45.066562 2027 raft_consensus.cc:504] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } }
I20240426 09:48:45.066920 2027 leader_election.cc:288] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [CANDIDATE]: Term 1 election: Requested vote from peers a2763d16993042bd966e8533f0ccba81 (127.0.99.133:38045), 83d022dc1b4e40af83870a08512c582a (127.0.99.130:44399)
I20240426 09:48:45.067245 1957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc5b6623dbd4f3c9a50d2da74344189" candidate_uuid: "3371a27570a94a02bcd4109e88a9fc6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2763d16993042bd966e8533f0ccba81"
W20240426 09:48:45.067289 2004 tablet.cc:2369] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:48:45.067299 1564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bc5b6623dbd4f3c9a50d2da74344189" candidate_uuid: "3371a27570a94a02bcd4109e88a9fc6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83d022dc1b4e40af83870a08512c582a"
I20240426 09:48:45.067370 1957 raft_consensus.cc:3037] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:45.067402 1564 raft_consensus.cc:3037] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:45.068307 1564 raft_consensus.cc:2454] T 6bc5b6623dbd4f3c9a50d2da74344189 P 83d022dc1b4e40af83870a08512c582a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3371a27570a94a02bcd4109e88a9fc6d in term 1.
I20240426 09:48:45.068315 1957 raft_consensus.cc:2454] T 6bc5b6623dbd4f3c9a50d2da74344189 P a2763d16993042bd966e8533f0ccba81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3371a27570a94a02bcd4109e88a9fc6d in term 1.
I20240426 09:48:45.068593 1631 leader_election.cc:302] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3371a27570a94a02bcd4109e88a9fc6d, a2763d16993042bd966e8533f0ccba81; no voters:
I20240426 09:48:45.068760 2027 raft_consensus.cc:2781] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:45.068990 2027 raft_consensus.cc:686] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [term 1 LEADER]: Becoming Leader. State: Replica: 3371a27570a94a02bcd4109e88a9fc6d, State: Running, Role: LEADER
I20240426 09:48:45.069283 2027 consensus_queue.cc:237] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } }
I20240426 09:48:45.072058 1290 catalog_manager.cc:5484] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3371a27570a94a02bcd4109e88a9fc6d (127.0.99.131). New cstate: current_term: 1 leader_uuid: "3371a27570a94a02bcd4109e88a9fc6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3371a27570a94a02bcd4109e88a9fc6d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38183 } } peers { permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 } } peers { permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 } } }
I20240426 09:48:45.084985 398 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240426 09:48:45.086274 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 139bff50b7a24569b04063fefa9c725f to finish bootstrapping
I20240426 09:48:45.091879 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 83d022dc1b4e40af83870a08512c582a to finish bootstrapping
I20240426 09:48:45.096509 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 3371a27570a94a02bcd4109e88a9fc6d to finish bootstrapping
I20240426 09:48:45.101007 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 582e0061fda545bb95efae811651d5d7 to finish bootstrapping
I20240426 09:48:45.106063 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver a2763d16993042bd966e8533f0ccba81 to finish bootstrapping
I20240426 09:48:45.110453 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 1351
I20240426 09:48:45.296605 1544 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.296746 1675 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:45.298652 1937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.303692 1806 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.373713 1675 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:45.374298 1544 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.374518 1937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.375353 1806 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.462695 2027 consensus_queue.cc:1035] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83d022dc1b4e40af83870a08512c582a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:48:45.473575 1937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.474534 1806 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.476255 1544 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.478942 1675 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:45.483392 2031 consensus_queue.cc:1035] T 6bc5b6623dbd4f3c9a50d2da74344189 P 3371a27570a94a02bcd4109e88a9fc6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2763d16993042bd966e8533f0ccba81" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 38045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:48:45.646703 1872 heartbeater.cc:503] Master 127.0.99.190:34561 was elected leader, sending a full tablet report...
I20240426 09:48:45.698457 1806 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.699810 1675 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:45.700363 1544 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.705932 1937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.771905 1675 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:45.772897 1806 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.773799 1937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.774137 1544 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.872066 1806 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.875929 1675 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:45.881631 1937 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.882510 1544 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:45.918887 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 1482
I20240426 09:48:45.926050 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 1613
I20240426 09:48:45.932253 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 1744
I20240426 09:48:45.937660 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 1875
I20240426 09:48:45.943533 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 1260
2024-04-26T09:48:45Z chronyd exiting
[ OK ] RebalanceStartCriteriaTest.TabletServerIsDown/0 (2280 ms)
[----------] 1 test from RebalanceStartCriteriaTest (2280 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/0
I20240426 09:48:45.967880 398 test_util.cc:268] Using random seed: -816833023
I20240426 09:48:45.968703 398 ts_itest-base.cc:112] Starting cluster with:
I20240426 09:48:45.968741 398 ts_itest-base.cc:113] --------------
I20240426 09:48:45.968752 398 ts_itest-base.cc:114] 7 tablet servers
I20240426 09:48:45.968763 398 ts_itest-base.cc:115] 1 replicas per TS
I20240426 09:48:45.968773 398 ts_itest-base.cc:116] --------------
2024-04-26T09:48:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-04-26T09:48:45Z Disabled control of system clock
I20240426 09:48:45.976538 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:33795
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:33795
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240426 09:48:46.087580 2225 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240426 09:48:46.087839 2225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:46.087913 2225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:46.087989 2225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:46.091113 2225 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:46.091199 2225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240426 09:48:46.091246 2225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:46.091281 2225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240426 09:48:46.091315 2225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240426 09:48:46.095099 2225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:33795
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:33795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.2225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:46.096073 2225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:46.097281 2225 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:46.105319 2233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:46.106056 2230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:46.106843 2231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:46.107610 2225 server_base.cc:1034] running on GCE node
I20240426 09:48:46.108127 2225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:46.109071 2225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:46.110239 2225 hybrid_clock.cc:648] HybridClock initialized: now 1714124926110230 us; error 32 us; skew 500 ppm
I20240426 09:48:46.112190 2225 webserver.cc:460] Webserver started at http://127.0.99.190:34273/ using document root <none> and password file <none>
I20240426 09:48:46.112741 2225 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:46.112820 2225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:46.112996 2225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:46.114626 2225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "7e29b764f7de462a82e49984b426f327"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.115165 2225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "7e29b764f7de462a82e49984b426f327"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.118484 2225 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240426 09:48:46.120710 2239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.121769 2225 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:46.121932 2225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "7e29b764f7de462a82e49984b426f327"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.122020 2225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:46.149317 2225 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:46.149979 2225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:46.150141 2225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:46.156553 2225 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:33795
I20240426 09:48:46.156566 2291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:33795 every 8 connection(s)
I20240426 09:48:46.157527 2225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240426 09:48:46.160111 2292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:46.165082 2292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327: Bootstrap starting.
I20240426 09:48:46.166582 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2225
I20240426 09:48:46.166676 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240426 09:48:46.167392 2292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:46.168275 2292 log.cc:824] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:46.170096 2292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327: No bootstrap required, opened a new log
I20240426 09:48:46.172883 2292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29b764f7de462a82e49984b426f327" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 33795 } }
I20240426 09:48:46.173110 2292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:46.173235 2292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e29b764f7de462a82e49984b426f327, State: Initialized, Role: FOLLOWER
I20240426 09:48:46.173722 2292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29b764f7de462a82e49984b426f327" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 33795 } }
I20240426 09:48:46.173885 2292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:46.173992 2292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:46.174113 2292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:46.270992 2292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29b764f7de462a82e49984b426f327" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 33795 } }
I20240426 09:48:46.271495 2292 leader_election.cc:302] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e29b764f7de462a82e49984b426f327; no voters:
I20240426 09:48:46.271901 2292 leader_election.cc:288] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:46.272312 2297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:46.272534 2297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [term 1 LEADER]: Becoming Leader. State: Replica: 7e29b764f7de462a82e49984b426f327, State: Running, Role: LEADER
I20240426 09:48:46.272877 2297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29b764f7de462a82e49984b426f327" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 33795 } }
I20240426 09:48:46.273169 2292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 09:48:46.274106 2297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e29b764f7de462a82e49984b426f327. Latest consensus state: current_term: 1 leader_uuid: "7e29b764f7de462a82e49984b426f327" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29b764f7de462a82e49984b426f327" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 33795 } } }
I20240426 09:48:46.274322 2297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [sys.catalog]: This master's current role is: LEADER
I20240426 09:48:46.274461 2298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e29b764f7de462a82e49984b426f327" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29b764f7de462a82e49984b426f327" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 33795 } } }
I20240426 09:48:46.274566 2298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327 [sys.catalog]: This master's current role is: LEADER
I20240426 09:48:46.274839 2306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 09:48:46.277788 2306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 09:48:46.283020 2306 catalog_manager.cc:1347] Generated new cluster ID: 845dffe3c8da4089a608d6774bafbac0
I20240426 09:48:46.283078 2306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 09:48:46.318941 2306 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 09:48:46.319881 2306 catalog_manager.cc:1504] Loading token signing keys...
I20240426 09:48:46.328406 2306 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 7e29b764f7de462a82e49984b426f327: Generated new TSK 0
I20240426 09:48:46.329294 2306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 09:48:46.338393 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:48:46.451273 2316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:46.451498 2316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:46.451545 2316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:46.454663 2316 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:46.454727 2316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:46.454850 2316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240426 09:48:46.458446 2316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.2316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:46.459450 2316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:46.460642 2316 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:46.470358 2324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:46.471004 2322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:46.471740 2321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:46.472558 2316 server_base.cc:1034] running on GCE node
I20240426 09:48:46.473006 2316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:46.473608 2316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:46.474851 2316 hybrid_clock.cc:648] HybridClock initialized: now 1714124926474833 us; error 38 us; skew 500 ppm
I20240426 09:48:46.476853 2316 webserver.cc:460] Webserver started at http://127.0.99.129:38215/ using document root <none> and password file <none>
I20240426 09:48:46.477427 2316 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:46.477489 2316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:46.477665 2316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:46.479393 2316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "506a29f8a6094b909536aa04d980679d"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.479853 2316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "506a29f8a6094b909536aa04d980679d"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.483126 2316 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240426 09:48:46.485257 2330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.486248 2316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:46.486356 2316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "506a29f8a6094b909536aa04d980679d"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.486438 2316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:46.503947 2316 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:46.504639 2316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:46.504799 2316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:46.505355 2316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:46.506238 2316 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:46.506289 2316 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.506345 2316 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:46.506377 2316 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.514488 2316 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:34009
I20240426 09:48:46.514518 2443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:34009 every 8 connection(s)
I20240426 09:48:46.515435 2316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240426 09:48:46.524003 2444 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:46.524223 2444 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:46.524675 2444 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:46.524685 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2316
I20240426 09:48:46.524783 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240426 09:48:46.526085 2256 ts_manager.cc:194] Registered new tserver with Master: 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:48:46.526744 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:46.527361 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:36715
W20240426 09:48:46.635800 2447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:46.636027 2447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:46.636075 2447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:46.639266 2447 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:46.639330 2447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:46.639434 2447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240426 09:48:46.643095 2447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.2447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:46.643976 2447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:46.645082 2447 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:46.653905 2452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:46.654726 2455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:46.655465 2453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:46.656204 2447 server_base.cc:1034] running on GCE node
I20240426 09:48:46.656556 2447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:46.657075 2447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:46.658233 2447 hybrid_clock.cc:648] HybridClock initialized: now 1714124926658225 us; error 23 us; skew 500 ppm
I20240426 09:48:46.660069 2447 webserver.cc:460] Webserver started at http://127.0.99.130:41169/ using document root <none> and password file <none>
I20240426 09:48:46.660616 2447 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:46.660669 2447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:46.660856 2447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:46.662469 2447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0ad0612beac445e896fcfc0562baa296"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.662950 2447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0ad0612beac445e896fcfc0562baa296"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.666118 2447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240426 09:48:46.668407 2461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.669374 2447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240426 09:48:46.669471 2447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0ad0612beac445e896fcfc0562baa296"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.669549 2447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:46.685982 2447 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:46.686570 2447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:46.686726 2447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:46.687302 2447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:46.688174 2447 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:46.688220 2447 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.688282 2447 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:46.688309 2447 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.696354 2447 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:38121
I20240426 09:48:46.696379 2574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:38121 every 8 connection(s)
I20240426 09:48:46.697234 2447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240426 09:48:46.703207 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2447
I20240426 09:48:46.703330 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240426 09:48:46.705257 2575 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:46.705341 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:46.705461 2575 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:46.705889 2575 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:46.706863 2256 ts_manager.cc:194] Registered new tserver with Master: 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:48:46.707391 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:52365
W20240426 09:48:46.812341 2578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:46.812582 2578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:46.812633 2578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:46.815747 2578 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:46.815829 2578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:46.815945 2578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240426 09:48:46.819725 2578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.2578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:46.820638 2578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:46.821799 2578 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:46.830700 2583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:46.831436 2584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:46.832223 2586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:46.833086 2578 server_base.cc:1034] running on GCE node
I20240426 09:48:46.833485 2578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:46.834105 2578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:46.835250 2578 hybrid_clock.cc:648] HybridClock initialized: now 1714124926835228 us; error 42 us; skew 500 ppm
I20240426 09:48:46.837229 2578 webserver.cc:460] Webserver started at http://127.0.99.131:34345/ using document root <none> and password file <none>
I20240426 09:48:46.837767 2578 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:46.837824 2578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:46.838001 2578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:46.839700 2578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "55f9bcefe16745108099bdd94114d98f"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.840219 2578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "55f9bcefe16745108099bdd94114d98f"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.843644 2578 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240426 09:48:46.845858 2592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.846956 2578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240426 09:48:46.847105 2578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "55f9bcefe16745108099bdd94114d98f"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:46.847195 2578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:46.866003 2578 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:46.866827 2578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:46.867019 2578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:46.867619 2578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:46.868616 2578 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:46.868666 2578 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.868731 2578 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:46.868759 2578 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:46.877336 2578 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:44151
I20240426 09:48:46.877349 2705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:44151 every 8 connection(s)
I20240426 09:48:46.878329 2578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240426 09:48:46.880956 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2578
I20240426 09:48:46.881065 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240426 09:48:46.883405 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:46.891106 2706 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:46.891338 2706 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:46.891786 2706 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:46.892774 2256 ts_manager.cc:194] Registered new tserver with Master: 55f9bcefe16745108099bdd94114d98f (127.0.99.131:44151)
I20240426 09:48:46.893534 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:57697
W20240426 09:48:46.995784 2709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:46.995996 2709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:46.996039 2709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:46.999792 2709 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:46.999864 2709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:46.999931 2709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240426 09:48:47.003619 2709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.2709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:47.004499 2709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:47.005574 2709 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:47.014446 2715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:47.015165 2717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:47.016016 2714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:47.016911 2709 server_base.cc:1034] running on GCE node
I20240426 09:48:47.017315 2709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:47.017864 2709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:47.019008 2709 hybrid_clock.cc:648] HybridClock initialized: now 1714124927018994 us; error 26 us; skew 500 ppm
I20240426 09:48:47.020936 2709 webserver.cc:460] Webserver started at http://127.0.99.132:45553/ using document root <none> and password file <none>
I20240426 09:48:47.021490 2709 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:47.021545 2709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:47.021732 2709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:47.023491 2709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "a18c2b34c33241afbd517065a5b797b6"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.023952 2709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "a18c2b34c33241afbd517065a5b797b6"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.027287 2709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240426 09:48:47.029346 2723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.030297 2709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:47.030401 2709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "a18c2b34c33241afbd517065a5b797b6"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.030481 2709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:47.051440 2709 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:47.052073 2709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:47.052248 2709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:47.052788 2709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:47.053654 2709 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:47.053704 2709 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.053766 2709 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:47.053791 2709 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.062245 2709 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:36485
I20240426 09:48:47.062269 2836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:36485 every 8 connection(s)
I20240426 09:48:47.063223 2709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240426 09:48:47.069736 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2709
I20240426 09:48:47.069851 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240426 09:48:47.072266 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:0
--local_ip_for_outbound_sockets=127.0.99.133
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:47.072548 2837 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:47.072829 2837 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:47.073341 2837 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:47.074530 2256 ts_manager.cc:194] Registered new tserver with Master: a18c2b34c33241afbd517065a5b797b6 (127.0.99.132:36485)
I20240426 09:48:47.075356 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:55695
W20240426 09:48:47.180590 2840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:47.180819 2840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:47.180897 2840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:47.183907 2840 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:47.183985 2840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:47.184082 2840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240426 09:48:47.187757 2840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.2840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:47.188660 2840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:47.189762 2840 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:47.198382 2848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:47.199259 2845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:47.199899 2846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:47.200776 2840 server_base.cc:1034] running on GCE node
I20240426 09:48:47.201179 2840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:47.201742 2840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:47.202895 2840 hybrid_clock.cc:648] HybridClock initialized: now 1714124927202876 us; error 29 us; skew 500 ppm
I20240426 09:48:47.204905 2840 webserver.cc:460] Webserver started at http://127.0.99.133:37233/ using document root <none> and password file <none>
I20240426 09:48:47.205482 2840 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:47.205556 2840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:47.205777 2840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:47.207592 2840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "400c68b80d144bd1b094c5cae03a111a"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.208065 2840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "400c68b80d144bd1b094c5cae03a111a"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.211409 2840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240426 09:48:47.213479 2854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.214452 2840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:47.214557 2840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "400c68b80d144bd1b094c5cae03a111a"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.214641 2840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:47.246433 2840 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:47.247121 2840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:47.247293 2840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:47.247823 2840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:47.248677 2840 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:47.248723 2840 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.248780 2840 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:47.248812 2840 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.257359 2840 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:45093
I20240426 09:48:47.257377 2967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:45093 every 8 connection(s)
I20240426 09:48:47.258379 2840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240426 09:48:47.266813 2968 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:47.267046 2968 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:47.267494 2968 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:47.268501 2256 ts_manager.cc:194] Registered new tserver with Master: 400c68b80d144bd1b094c5cae03a111a (127.0.99.133:45093)
I20240426 09:48:47.268657 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2840
I20240426 09:48:47.268776 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240426 09:48:47.269115 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:39671
I20240426 09:48:47.270956 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:0
--local_ip_for_outbound_sockets=127.0.99.134
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:48:47.387300 2971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:47.387530 2971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:47.387576 2971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:47.390828 2971 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:47.390900 2971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:47.390987 2971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240426 09:48:47.394663 2971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.2971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:47.395700 2971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:47.396775 2971 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:47.405388 2977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:47.406132 2976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:47.407013 2979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:47.407838 2971 server_base.cc:1034] running on GCE node
I20240426 09:48:47.408238 2971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:47.408918 2971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:47.410069 2971 hybrid_clock.cc:648] HybridClock initialized: now 1714124927410056 us; error 29 us; skew 500 ppm
I20240426 09:48:47.412019 2971 webserver.cc:460] Webserver started at http://127.0.99.134:37627/ using document root <none> and password file <none>
I20240426 09:48:47.412586 2971 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:47.412648 2971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:47.412828 2971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:47.414462 2971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "010c57b7730449c3b8c982a057e6268b"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.414971 2971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "010c57b7730449c3b8c982a057e6268b"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.418309 2971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20240426 09:48:47.420457 2985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.421433 2971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:48:47.421533 2971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "010c57b7730449c3b8c982a057e6268b"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.421613 2971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:47.434094 2971 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:47.434686 2971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:47.434897 2971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:47.435432 2971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:47.436316 2971 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:47.436362 2971 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.436407 2971 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:47.436439 2971 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.444974 2971 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:41855
I20240426 09:48:47.444993 3098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:41855 every 8 connection(s)
I20240426 09:48:47.445922 2971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240426 09:48:47.447018 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2971
I20240426 09:48:47.447130 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240426 09:48:47.449309 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:0
--local_ip_for_outbound_sockets=127.0.99.135
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:47.455847 3099 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:47.456115 3099 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:47.456733 3099 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:47.457836 2256 ts_manager.cc:194] Registered new tserver with Master: 010c57b7730449c3b8c982a057e6268b (127.0.99.134:41855)
I20240426 09:48:47.458633 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:52387
I20240426 09:48:47.529518 2444 heartbeater.cc:503] Master 127.0.99.190:33795 was elected leader, sending a full tablet report...
W20240426 09:48:47.564081 3102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:47.564303 3102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:47.564347 3102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:47.567337 3102 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:47.567440 3102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:47.567589 3102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240426 09:48:47.571367 3102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.3102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:47.572270 3102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:47.573457 3102 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:47.582604 3110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:47.583364 3107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:47.584188 3108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:47.585012 3102 server_base.cc:1034] running on GCE node
I20240426 09:48:47.585408 3102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:47.585955 3102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:47.587097 3102 hybrid_clock.cc:648] HybridClock initialized: now 1714124927587085 us; error 30 us; skew 500 ppm
I20240426 09:48:47.589782 3102 webserver.cc:460] Webserver started at http://127.0.99.135:36559/ using document root <none> and password file <none>
I20240426 09:48:47.590420 3102 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:47.590484 3102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:47.590672 3102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:48:47.592429 3102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "3b591a3ffd514de7959be53199789ce8"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.592877 3102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "3b591a3ffd514de7959be53199789ce8"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.596150 3102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20240426 09:48:47.598199 3116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.599228 3102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:47.599329 3102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "3b591a3ffd514de7959be53199789ce8"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:47.599409 3102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:47.610517 3102 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:47.611114 3102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:47.611286 3102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:47.611776 3102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:47.612644 3102 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:47.612689 3102 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.612742 3102 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:47.612772 3102 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:47.622596 3102 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:33487
I20240426 09:48:47.622624 3229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:33487 every 8 connection(s)
I20240426 09:48:47.623637 3102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240426 09:48:47.625500 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 3102
I20240426 09:48:47.625623 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240426 09:48:47.632488 3230 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:47.632697 3230 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:47.633131 3230 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:47.634056 2256 ts_manager.cc:194] Registered new tserver with Master: 3b591a3ffd514de7959be53199789ce8 (127.0.99.135:33487)
I20240426 09:48:47.634588 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:36835
I20240426 09:48:47.639663 398 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240426 09:48:47.648569 2256 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56950:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240426 09:48:47.665495 2640 tablet_service.cc:1467] Processing CreateTablet for tablet 31de0dcd76d94411ad4a0e2968960f26 (DEFAULT_TABLE table=TestTable [id=d7b2f4254f6a4f4ebdda5ef2457aa8db]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:47.666460 2640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31de0dcd76d94411ad4a0e2968960f26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:47.671903 3244 tablet_bootstrap.cc:492] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Bootstrap starting.
I20240426 09:48:47.674605 3244 tablet_bootstrap.cc:654] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:47.675469 3244 log.cc:824] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:47.677340 3244 tablet_bootstrap.cc:492] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: No bootstrap required, opened a new log
I20240426 09:48:47.677525 3244 ts_tablet_manager.cc:1367] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20240426 09:48:47.681631 3244 raft_consensus.cc:348] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:48:47.681835 3244 raft_consensus.cc:374] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:47.681878 3244 raft_consensus.cc:729] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f9bcefe16745108099bdd94114d98f, State: Initialized, Role: FOLLOWER
I20240426 09:48:47.682297 3244 consensus_queue.cc:260] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:48:47.682417 3244 raft_consensus.cc:388] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:47.682466 3244 raft_consensus.cc:482] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:47.682546 3244 raft_consensus.cc:3037] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:47.709646 2575 heartbeater.cc:503] Master 127.0.99.190:33795 was elected leader, sending a full tablet report...
I20240426 09:48:47.777753 3244 raft_consensus.cc:504] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:48:47.778370 3244 leader_election.cc:302] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55f9bcefe16745108099bdd94114d98f; no voters:
I20240426 09:48:47.778920 3244 leader_election.cc:288] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:47.779322 3246 raft_consensus.cc:2781] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:47.779554 3246 raft_consensus.cc:686] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 1 LEADER]: Becoming Leader. State: Replica: 55f9bcefe16745108099bdd94114d98f, State: Running, Role: LEADER
I20240426 09:48:47.779794 2706 heartbeater.cc:503] Master 127.0.99.190:33795 was elected leader, sending a full tablet report...
I20240426 09:48:47.779922 3246 consensus_queue.cc:237] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:48:47.780078 3244 ts_tablet_manager.cc:1398] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Time spent starting tablet: real 0.102s user 0.008s sys 0.001s
I20240426 09:48:47.783334 2256 catalog_manager.cc:5484] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f reported cstate change: term changed from 0 to 1, leader changed from <none> to 55f9bcefe16745108099bdd94114d98f (127.0.99.131). New cstate: current_term: 1 leader_uuid: "55f9bcefe16745108099bdd94114d98f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } } }
I20240426 09:48:47.811947 398 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240426 09:48:47.813277 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 506a29f8a6094b909536aa04d980679d to finish bootstrapping
I20240426 09:48:47.819183 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 0ad0612beac445e896fcfc0562baa296 to finish bootstrapping
I20240426 09:48:47.824683 398 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 55f9bcefe16745108099bdd94114d98f to finish bootstrapping
I20240426 09:48:47.829799 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver a18c2b34c33241afbd517065a5b797b6 to finish bootstrapping
I20240426 09:48:47.835369 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 400c68b80d144bd1b094c5cae03a111a to finish bootstrapping
I20240426 09:48:47.840806 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 010c57b7730449c3b8c982a057e6268b to finish bootstrapping
I20240426 09:48:47.846357 398 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 3b591a3ffd514de7959be53199789ce8 to finish bootstrapping
I20240426 09:48:47.851642 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 2578
I20240426 09:48:47.857750 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 2709
I20240426 09:48:47.864229 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 2840
I20240426 09:48:47.869928 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 2971
I20240426 09:48:47.876076 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 3102
I20240426 09:48:51.633944 2256 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56950:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:48:51.645403 2509 tablet_service.cc:1467] Processing CreateTablet for tablet 248ceb0acd6c445fb2644fcc2c4f5b55 (DEFAULT_TABLE table=rebalance_test_table_0 [id=d357ae9b5bc84013aa8876bfd166273a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:51.645403 2377 tablet_service.cc:1467] Processing CreateTablet for tablet 87ca4c907ced49ecb4b8acc48efe734d (DEFAULT_TABLE table=rebalance_test_table_0 [id=d357ae9b5bc84013aa8876bfd166273a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:51.645403 2378 tablet_service.cc:1467] Processing CreateTablet for tablet 1900ef96bb49402c9c757918f7fe3a41 (DEFAULT_TABLE table=rebalance_test_table_0 [id=d357ae9b5bc84013aa8876bfd166273a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:51.646368 2378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1900ef96bb49402c9c757918f7fe3a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:51.646369 2509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248ceb0acd6c445fb2644fcc2c4f5b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:51.646747 2377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ca4c907ced49ecb4b8acc48efe734d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:51.651954 3266 tablet_bootstrap.cc:492] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d: Bootstrap starting.
I20240426 09:48:51.652385 3267 tablet_bootstrap.cc:492] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296: Bootstrap starting.
I20240426 09:48:51.654009 3266 tablet_bootstrap.cc:654] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:51.654187 3267 tablet_bootstrap.cc:654] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:51.654659 3266 log.cc:824] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:51.654870 3267 log.cc:824] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:51.656152 3266 tablet_bootstrap.cc:492] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d: No bootstrap required, opened a new log
I20240426 09:48:51.656294 3267 tablet_bootstrap.cc:492] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296: No bootstrap required, opened a new log
I20240426 09:48:51.656313 3266 ts_tablet_manager.cc:1367] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240426 09:48:51.656451 3267 ts_tablet_manager.cc:1367] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240426 09:48:51.659317 3266 raft_consensus.cc:348] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.659500 3266 raft_consensus.cc:374] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:51.659572 3266 raft_consensus.cc:729] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Initialized, Role: FOLLOWER
I20240426 09:48:51.659556 3267 raft_consensus.cc:348] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.659797 3267 raft_consensus.cc:374] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:51.659898 3267 raft_consensus.cc:729] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Initialized, Role: FOLLOWER
I20240426 09:48:51.659977 3266 consensus_queue.cc:260] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.660140 3266 raft_consensus.cc:388] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:51.660204 3266 raft_consensus.cc:482] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:51.660336 3266 raft_consensus.cc:3037] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:51.660337 3267 consensus_queue.cc:260] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.660501 3267 raft_consensus.cc:388] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:51.660569 3267 raft_consensus.cc:482] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:51.660672 3267 raft_consensus.cc:3037] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:51.661178 3266 raft_consensus.cc:504] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.661470 3266 leader_election.cc:302] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:48:51.661631 3267 raft_consensus.cc:504] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.661736 3266 leader_election.cc:288] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:51.661818 3270 raft_consensus.cc:2781] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:51.661947 3267 leader_election.cc:302] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296; no voters:
I20240426 09:48:51.662003 3270 raft_consensus.cc:686] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Leader. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:48:51.662302 3267 leader_election.cc:288] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:51.662279 3270 consensus_queue.cc:237] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.662616 3274 raft_consensus.cc:2781] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:51.662839 3274 raft_consensus.cc:686] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:48:51.662890 3266 ts_tablet_manager.cc:1398] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20240426 09:48:51.663102 3267 ts_tablet_manager.cc:1398] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20240426 09:48:51.663131 3274 consensus_queue.cc:237] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.663192 3266 tablet_bootstrap.cc:492] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: Bootstrap starting.
I20240426 09:48:51.664997 2256 catalog_manager.cc:5484] T 1900ef96bb49402c9c757918f7fe3a41 P 506a29f8a6094b909536aa04d980679d reported cstate change: term changed from 0 to 1, leader changed from <none> to 506a29f8a6094b909536aa04d980679d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } }
I20240426 09:48:51.665237 3266 tablet_bootstrap.cc:654] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:51.665848 2255 catalog_manager.cc:5484] T 248ceb0acd6c445fb2644fcc2c4f5b55 P 0ad0612beac445e896fcfc0562baa296 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad0612beac445e896fcfc0562baa296 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } }
I20240426 09:48:51.668035 3266 tablet_bootstrap.cc:492] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: No bootstrap required, opened a new log
I20240426 09:48:51.668123 3266 ts_tablet_manager.cc:1367] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:48:51.668648 3266 raft_consensus.cc:348] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.668799 3266 raft_consensus.cc:374] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:51.668848 3266 raft_consensus.cc:729] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Initialized, Role: FOLLOWER
I20240426 09:48:51.669001 3266 consensus_queue.cc:260] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.669095 3266 raft_consensus.cc:388] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:51.669160 3266 raft_consensus.cc:482] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:51.669211 3266 raft_consensus.cc:3037] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:51.669890 3266 raft_consensus.cc:504] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.670002 3266 leader_election.cc:302] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:48:51.670099 3266 leader_election.cc:288] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:51.670204 3272 raft_consensus.cc:2781] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:51.670310 3272 raft_consensus.cc:686] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Leader. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:48:51.670336 3266 ts_tablet_manager.cc:1398] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:51.670408 3272 consensus_queue.cc:237] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.671509 2255 catalog_manager.cc:5484] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d reported cstate change: term changed from 0 to 1, leader changed from <none> to 506a29f8a6094b909536aa04d980679d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } }
W20240426 09:48:51.786438 3282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:51.786685 3282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:51.789938 3282 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
INSERT report
rows total: 2000
time total: 58.7052 ms
time per row: 0.0293526 ms
I20240426 09:48:51.893564 2256 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56950:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:48:51.898869 2509 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef96c18a0c8468e8d7668e05d7c4eea (DEFAULT_TABLE table=rebalance_test_table_1 [id=4f3ba36f2bfb4b28991e7efdbfe5e664]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:51.898869 2508 tablet_service.cc:1467] Processing CreateTablet for tablet 388e1882272944a3893813caef6cd903 (DEFAULT_TABLE table=rebalance_test_table_1 [id=4f3ba36f2bfb4b28991e7efdbfe5e664]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:51.898979 2378 tablet_service.cc:1467] Processing CreateTablet for tablet 54dd16182f294fe9af3ff82cd57b7e5a (DEFAULT_TABLE table=rebalance_test_table_1 [id=4f3ba36f2bfb4b28991e7efdbfe5e664]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:51.899266 2509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ef96c18a0c8468e8d7668e05d7c4eea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:51.899292 2378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54dd16182f294fe9af3ff82cd57b7e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:51.899430 2508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388e1882272944a3893813caef6cd903. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:51.901369 3266 tablet_bootstrap.cc:492] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: Bootstrap starting.
I20240426 09:48:51.901574 3267 tablet_bootstrap.cc:492] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296: Bootstrap starting.
I20240426 09:48:51.902297 3266 tablet_bootstrap.cc:654] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:51.902379 3267 tablet_bootstrap.cc:654] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:51.903340 3266 tablet_bootstrap.cc:492] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: No bootstrap required, opened a new log
I20240426 09:48:51.903390 3267 tablet_bootstrap.cc:492] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296: No bootstrap required, opened a new log
I20240426 09:48:51.903422 3266 ts_tablet_manager.cc:1367] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:48:51.903457 3267 ts_tablet_manager.cc:1367] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:48:51.903802 3266 raft_consensus.cc:348] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.903901 3266 raft_consensus.cc:374] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:51.903901 3267 raft_consensus.cc:348] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.903934 3266 raft_consensus.cc:729] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Initialized, Role: FOLLOWER
I20240426 09:48:51.903977 3267 raft_consensus.cc:374] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:51.904026 3267 raft_consensus.cc:729] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Initialized, Role: FOLLOWER
I20240426 09:48:51.904068 3266 consensus_queue.cc:260] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.904135 3266 raft_consensus.cc:388] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:51.904167 3266 raft_consensus.cc:482] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:51.904151 3267 consensus_queue.cc:260] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.904202 3266 raft_consensus.cc:3037] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:51.904225 3267 raft_consensus.cc:388] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:51.904248 3267 raft_consensus.cc:482] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:51.904276 3267 raft_consensus.cc:3037] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:51.904776 3266 raft_consensus.cc:504] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.904815 3267 raft_consensus.cc:504] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.904868 3266 leader_election.cc:302] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:48:51.904899 3267 leader_election.cc:302] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296; no voters:
I20240426 09:48:51.904937 3266 leader_election.cc:288] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:51.904968 3267 leader_election.cc:288] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:51.904999 3272 raft_consensus.cc:2781] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:51.905045 3275 raft_consensus.cc:2781] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:51.905061 3272 raft_consensus.cc:686] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Leader. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:48:51.905156 3272 consensus_queue.cc:237] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:51.905388 3275 raft_consensus.cc:686] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:48:51.905412 3267 ts_tablet_manager.cc:1398] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:51.905525 3275 consensus_queue.cc:237] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.905628 3267 tablet_bootstrap.cc:492] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: Bootstrap starting.
I20240426 09:48:51.906567 3267 tablet_bootstrap.cc:654] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:51.905166 3266 ts_tablet_manager.cc:1398] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:48:51.906659 2256 catalog_manager.cc:5484] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d reported cstate change: term changed from 0 to 1, leader changed from <none> to 506a29f8a6094b909536aa04d980679d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } }
I20240426 09:48:51.907318 2254 catalog_manager.cc:5484] T 1ef96c18a0c8468e8d7668e05d7c4eea P 0ad0612beac445e896fcfc0562baa296 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad0612beac445e896fcfc0562baa296 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } }
I20240426 09:48:51.908922 3267 tablet_bootstrap.cc:492] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: No bootstrap required, opened a new log
I20240426 09:48:51.909034 3267 ts_tablet_manager.cc:1367] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:48:51.909472 3267 raft_consensus.cc:348] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.909567 3267 raft_consensus.cc:374] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:51.909637 3267 raft_consensus.cc:729] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Initialized, Role: FOLLOWER
I20240426 09:48:51.909765 3267 consensus_queue.cc:260] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.909847 3267 raft_consensus.cc:388] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:51.909924 3267 raft_consensus.cc:482] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:51.909974 3267 raft_consensus.cc:3037] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:51.910717 3267 raft_consensus.cc:504] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.910899 3267 leader_election.cc:302] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296; no voters:
I20240426 09:48:51.911008 3267 leader_election.cc:288] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:51.911046 3274 raft_consensus.cc:2781] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:51.911125 3274 raft_consensus.cc:686] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:48:51.911219 3274 consensus_queue.cc:237] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:51.911265 3267 ts_tablet_manager.cc:1398] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240426 09:48:51.912294 2252 catalog_manager.cc:5484] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad0612beac445e896fcfc0562baa296 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } }
W20240426 09:48:52.022249 3325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:52.022517 3325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:52.025519 3325 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
INSERT report
rows total: 2000
time total: 56.89 ms
time per row: 0.028445 ms
I20240426 09:48:52.124032 2252 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56950:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:48:52.129371 2377 tablet_service.cc:1467] Processing CreateTablet for tablet 80ada1082b6842f59c830591b94bd0e5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b49d74ebc4dc4e349407e21ef058b318]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:52.129441 2508 tablet_service.cc:1467] Processing CreateTablet for tablet 43b0427d465142a1b7639dd269112b15 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b49d74ebc4dc4e349407e21ef058b318]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:52.129372 2378 tablet_service.cc:1467] Processing CreateTablet for tablet faeca7ce592f4ffe806903cb915c75d4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=b49d74ebc4dc4e349407e21ef058b318]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:52.129747 2508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43b0427d465142a1b7639dd269112b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:52.129756 2377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80ada1082b6842f59c830591b94bd0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:52.129963 2378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faeca7ce592f4ffe806903cb915c75d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:52.131551 3266 tablet_bootstrap.cc:492] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d: Bootstrap starting.
I20240426 09:48:52.131853 3267 tablet_bootstrap.cc:492] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: Bootstrap starting.
I20240426 09:48:52.132475 3266 tablet_bootstrap.cc:654] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:52.132733 3267 tablet_bootstrap.cc:654] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:52.133782 3266 tablet_bootstrap.cc:492] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d: No bootstrap required, opened a new log
I20240426 09:48:52.133873 3266 ts_tablet_manager.cc:1367] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240426 09:48:52.134294 3266 raft_consensus.cc:348] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.134382 3266 raft_consensus.cc:374] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:52.134415 3266 raft_consensus.cc:729] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Initialized, Role: FOLLOWER
I20240426 09:48:52.134534 3266 consensus_queue.cc:260] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.134596 3266 raft_consensus.cc:388] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:52.134631 3266 raft_consensus.cc:482] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:52.134678 3266 raft_consensus.cc:3037] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:52.135425 3267 tablet_bootstrap.cc:492] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: No bootstrap required, opened a new log
I20240426 09:48:52.135520 3267 ts_tablet_manager.cc:1367] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:48:52.135541 3266 raft_consensus.cc:504] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.135679 3266 leader_election.cc:302] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:48:52.135771 3266 leader_election.cc:288] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:52.135835 3271 raft_consensus.cc:2781] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:52.135936 3271 raft_consensus.cc:686] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Leader. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:48:52.136013 3266 ts_tablet_manager.cc:1398] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:52.135993 3267 raft_consensus.cc:348] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.136111 3267 raft_consensus.cc:374] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:52.136065 3271 consensus_queue.cc:237] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.136154 3267 raft_consensus.cc:729] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Initialized, Role: FOLLOWER
I20240426 09:48:52.136276 3267 consensus_queue.cc:260] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.136366 3267 raft_consensus.cc:388] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:52.136404 3267 raft_consensus.cc:482] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:52.136461 3267 raft_consensus.cc:3037] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:52.137456 3267 raft_consensus.cc:504] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.137606 3267 leader_election.cc:302] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296; no voters:
I20240426 09:48:52.138216 3273 raft_consensus.cc:2781] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:52.138312 3273 raft_consensus.cc:686] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:48:52.138443 3273 consensus_queue.cc:237] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.139020 3267 leader_election.cc:288] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:52.139374 3267 ts_tablet_manager.cc:1398] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20240426 09:48:52.140180 2252 catalog_manager.cc:5484] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad0612beac445e896fcfc0562baa296 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } }
I20240426 09:48:52.140681 3266 tablet_bootstrap.cc:492] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: Bootstrap starting.
I20240426 09:48:52.141621 3266 tablet_bootstrap.cc:654] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:52.142863 3266 tablet_bootstrap.cc:492] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: No bootstrap required, opened a new log
I20240426 09:48:52.142947 3266 ts_tablet_manager.cc:1367] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:52.143446 3266 raft_consensus.cc:348] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.143576 3266 raft_consensus.cc:374] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:52.143616 3266 raft_consensus.cc:729] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Initialized, Role: FOLLOWER
I20240426 09:48:52.143658 2256 catalog_manager.cc:5484] T 80ada1082b6842f59c830591b94bd0e5 P 506a29f8a6094b909536aa04d980679d reported cstate change: term changed from 0 to 1, leader changed from <none> to 506a29f8a6094b909536aa04d980679d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } }
I20240426 09:48:52.143728 3266 consensus_queue.cc:260] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.143810 3266 raft_consensus.cc:388] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:52.143844 3266 raft_consensus.cc:482] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:52.143882 3266 raft_consensus.cc:3037] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:52.144767 3266 raft_consensus.cc:504] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.144904 3266 leader_election.cc:302] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:48:52.145000 3266 leader_election.cc:288] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:52.145138 3270 raft_consensus.cc:2781] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:52.145228 3270 raft_consensus.cc:686] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Leader. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:48:52.145344 3270 consensus_queue.cc:237] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.145231 3266 ts_tablet_manager.cc:1398] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240426 09:48:52.146739 2252 catalog_manager.cc:5484] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d reported cstate change: term changed from 0 to 1, leader changed from <none> to 506a29f8a6094b909536aa04d980679d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } }
W20240426 09:48:52.260613 3359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:52.260850 3359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:52.263752 3359 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
INSERT report
rows total: 2000
time total: 55.0023 ms
time per row: 0.0275011 ms
I20240426 09:48:52.358969 2252 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56950:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:48:52.364305 2509 tablet_service.cc:1467] Processing CreateTablet for tablet 469ea537fdbc46d8982f23b6a1a181ab (DEFAULT_TABLE table=rebalance_test_table_3 [id=42be3da92ccd4b128a37f478a31c77bc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:52.364478 2378 tablet_service.cc:1467] Processing CreateTablet for tablet 60af15bf6ca4441e9718d44380392ff0 (DEFAULT_TABLE table=rebalance_test_table_3 [id=42be3da92ccd4b128a37f478a31c77bc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:52.364675 2509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 469ea537fdbc46d8982f23b6a1a181ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:52.364779 2378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60af15bf6ca4441e9718d44380392ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:52.364305 2508 tablet_service.cc:1467] Processing CreateTablet for tablet 7de16751060b4b91b679ba22803ae980 (DEFAULT_TABLE table=rebalance_test_table_3 [id=42be3da92ccd4b128a37f478a31c77bc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:52.364943 2508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7de16751060b4b91b679ba22803ae980. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:52.366535 3267 tablet_bootstrap.cc:492] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: Bootstrap starting.
I20240426 09:48:52.366874 3266 tablet_bootstrap.cc:492] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: Bootstrap starting.
I20240426 09:48:52.367554 3267 tablet_bootstrap.cc:654] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:52.367744 3266 tablet_bootstrap.cc:654] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:52.368587 3267 tablet_bootstrap.cc:492] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: No bootstrap required, opened a new log
I20240426 09:48:52.368654 3266 tablet_bootstrap.cc:492] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: No bootstrap required, opened a new log
I20240426 09:48:52.368667 3267 ts_tablet_manager.cc:1367] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:52.368724 3266 ts_tablet_manager.cc:1367] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:48:52.369056 3267 raft_consensus.cc:348] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.369097 3266 raft_consensus.cc:348] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.369143 3267 raft_consensus.cc:374] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:52.369169 3266 raft_consensus.cc:374] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:52.369179 3267 raft_consensus.cc:729] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Initialized, Role: FOLLOWER
I20240426 09:48:52.369212 3266 raft_consensus.cc:729] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Initialized, Role: FOLLOWER
I20240426 09:48:52.369307 3267 consensus_queue.cc:260] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.369334 3266 consensus_queue.cc:260] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.369391 3266 raft_consensus.cc:388] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:52.369397 3267 raft_consensus.cc:388] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:52.369414 3266 raft_consensus.cc:482] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:52.369441 3267 raft_consensus.cc:482] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:52.369465 3266 raft_consensus.cc:3037] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:52.369488 3267 raft_consensus.cc:3037] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:52.370167 3267 raft_consensus.cc:504] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.370187 3266 raft_consensus.cc:504] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.370285 3267 leader_election.cc:302] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296; no voters:
I20240426 09:48:52.370301 3266 leader_election.cc:302] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:48:52.370429 3266 leader_election.cc:288] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:52.370466 3274 raft_consensus.cc:2781] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:52.370497 3272 raft_consensus.cc:2781] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:52.370541 3274 raft_consensus.cc:686] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:48:52.370604 3272 raft_consensus.cc:686] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Leader. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:48:52.370652 3274 consensus_queue.cc:237] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.370718 3272 consensus_queue.cc:237] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.370752 3266 ts_tablet_manager.cc:1398] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:52.371112 3267 leader_election.cc:288] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:52.371402 3267 ts_tablet_manager.cc:1398] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:48:52.371565 3267 tablet_bootstrap.cc:492] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: Bootstrap starting.
I20240426 09:48:52.372069 2252 catalog_manager.cc:5484] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad0612beac445e896fcfc0562baa296 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } }
I20240426 09:48:52.372102 2253 catalog_manager.cc:5484] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d reported cstate change: term changed from 0 to 1, leader changed from <none> to 506a29f8a6094b909536aa04d980679d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } }
I20240426 09:48:52.372545 3267 tablet_bootstrap.cc:654] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:52.373601 3267 tablet_bootstrap.cc:492] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: No bootstrap required, opened a new log
I20240426 09:48:52.373677 3267 ts_tablet_manager.cc:1367] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:48:52.374112 3267 raft_consensus.cc:348] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.374210 3267 raft_consensus.cc:374] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:52.374240 3267 raft_consensus.cc:729] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Initialized, Role: FOLLOWER
I20240426 09:48:52.374382 3267 consensus_queue.cc:260] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.374459 3267 raft_consensus.cc:388] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:52.374493 3267 raft_consensus.cc:482] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:52.374521 3267 raft_consensus.cc:3037] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:52.375304 3267 raft_consensus.cc:504] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.375430 3267 leader_election.cc:302] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296; no voters:
I20240426 09:48:52.375522 3267 leader_election.cc:288] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:52.375650 3274 raft_consensus.cc:2781] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:52.375739 3267 ts_tablet_manager.cc:1398] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:52.375761 3274 raft_consensus.cc:686] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:48:52.375857 3274 consensus_queue.cc:237] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.379384 2252 catalog_manager.cc:5484] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad0612beac445e896fcfc0562baa296 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } }
W20240426 09:48:52.487139 3390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:52.487371 3390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:52.490275 3390 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
INSERT report
rows total: 2000
time total: 58.9715 ms
time per row: 0.0294857 ms
I20240426 09:48:52.589728 2254 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56950:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:48:52.594986 2509 tablet_service.cc:1467] Processing CreateTablet for tablet 62547c20121048a09eda5cf3014bbff6 (DEFAULT_TABLE table=rebalance_test_table_4 [id=994cca768a73422ab7316a1feafaebfb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:52.594956 2378 tablet_service.cc:1467] Processing CreateTablet for tablet c4810d8df46c4404875a4a53a50ecb60 (DEFAULT_TABLE table=rebalance_test_table_4 [id=994cca768a73422ab7316a1feafaebfb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:52.595278 2509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62547c20121048a09eda5cf3014bbff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:52.595278 2378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4810d8df46c4404875a4a53a50ecb60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:52.595228 2377 tablet_service.cc:1467] Processing CreateTablet for tablet 0db2ff8ccef842a2ac866f85b0e05134 (DEFAULT_TABLE table=rebalance_test_table_4 [id=994cca768a73422ab7316a1feafaebfb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:48:52.595608 2377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db2ff8ccef842a2ac866f85b0e05134. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:52.597254 3266 tablet_bootstrap.cc:492] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: Bootstrap starting.
I20240426 09:48:52.597761 3267 tablet_bootstrap.cc:492] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: Bootstrap starting.
I20240426 09:48:52.598197 3266 tablet_bootstrap.cc:654] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:52.598803 3267 tablet_bootstrap.cc:654] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:52.599431 3266 tablet_bootstrap.cc:492] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: No bootstrap required, opened a new log
I20240426 09:48:52.599521 3266 ts_tablet_manager.cc:1367] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:48:52.599949 3267 tablet_bootstrap.cc:492] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: No bootstrap required, opened a new log
I20240426 09:48:52.599977 3266 raft_consensus.cc:348] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.600040 3267 ts_tablet_manager.cc:1367] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:52.600080 3266 raft_consensus.cc:374] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:52.600118 3266 raft_consensus.cc:729] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Initialized, Role: FOLLOWER
I20240426 09:48:52.600255 3266 consensus_queue.cc:260] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.600347 3266 raft_consensus.cc:388] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:52.600390 3266 raft_consensus.cc:482] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:52.600437 3266 raft_consensus.cc:3037] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:52.600509 3267 raft_consensus.cc:348] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.600618 3267 raft_consensus.cc:374] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:52.600661 3267 raft_consensus.cc:729] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Initialized, Role: FOLLOWER
I20240426 09:48:52.600769 3267 consensus_queue.cc:260] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.600843 3267 raft_consensus.cc:388] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:52.600878 3267 raft_consensus.cc:482] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:52.600919 3267 raft_consensus.cc:3037] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:52.601308 3266 raft_consensus.cc:504] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.601445 3266 leader_election.cc:302] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:48:52.601536 3266 leader_election.cc:288] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:52.601588 3272 raft_consensus.cc:2781] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:52.601696 3272 raft_consensus.cc:686] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Leader. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:48:52.601819 3272 consensus_queue.cc:237] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.601869 3267 raft_consensus.cc:504] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.601922 3266 ts_tablet_manager.cc:1398] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:48:52.602010 3267 leader_election.cc:302] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296; no voters:
I20240426 09:48:52.602058 3266 tablet_bootstrap.cc:492] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: Bootstrap starting.
I20240426 09:48:52.602151 3275 raft_consensus.cc:2781] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:52.602218 3275 raft_consensus.cc:686] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:48:52.602317 3275 consensus_queue.cc:237] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:52.603430 3266 tablet_bootstrap.cc:654] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: Neither blocks nor log segments found. Creating new log.
I20240426 09:48:52.603423 2254 catalog_manager.cc:5484] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d reported cstate change: term changed from 0 to 1, leader changed from <none> to 506a29f8a6094b909536aa04d980679d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } }
I20240426 09:48:52.604539 3266 tablet_bootstrap.cc:492] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: No bootstrap required, opened a new log
I20240426 09:48:52.604528 3267 leader_election.cc:288] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:52.604611 3266 ts_tablet_manager.cc:1367] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:48:52.604725 2253 catalog_manager.cc:5484] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad0612beac445e896fcfc0562baa296 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } }
I20240426 09:48:52.604843 3267 ts_tablet_manager.cc:1398] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20240426 09:48:52.605036 3266 raft_consensus.cc:348] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.605114 3266 raft_consensus.cc:374] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:48:52.605142 3266 raft_consensus.cc:729] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Initialized, Role: FOLLOWER
I20240426 09:48:52.605276 3266 consensus_queue.cc:260] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.605350 3266 raft_consensus.cc:388] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:52.605387 3266 raft_consensus.cc:482] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:52.605429 3266 raft_consensus.cc:3037] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:48:52.606271 3266 raft_consensus.cc:504] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.606396 3266 leader_election.cc:302] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:48:52.606487 3266 leader_election.cc:288] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:48:52.606551 3272 raft_consensus.cc:2781] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:48:52.606653 3272 raft_consensus.cc:686] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Leader. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:48:52.606724 3266 ts_tablet_manager.cc:1398] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20240426 09:48:52.606755 3272 consensus_queue.cc:237] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:52.608003 2253 catalog_manager.cc:5484] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d reported cstate change: term changed from 0 to 1, leader changed from <none> to 506a29f8a6094b909536aa04d980679d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } }
W20240426 09:48:52.717200 3422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:52.717474 3422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:52.720395 3422 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
INSERT report
rows total: 2000
time total: 70.8583 ms
time per row: 0.0354292 ms
I20240426 09:48:52.830825 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:44151
--local_ip_for_outbound_sockets=127.0.99.131
--tserver_master_addrs=127.0.99.190:33795
--webserver_port=34345
--webserver_interface=127.0.99.131
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:48:52.936542 3448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:52.936748 3448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:52.936789 3448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:52.939793 3448 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:52.939860 3448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:52.939941 3448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240426 09:48:52.943600 3448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:44151
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=34345
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.3448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:52.944619 3448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:52.945751 3448 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:52.954849 3453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:52.955593 3456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:52.956451 3454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:52.957299 3448 server_base.cc:1034] running on GCE node
I20240426 09:48:52.957696 3448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:52.958240 3448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:52.959388 3448 hybrid_clock.cc:648] HybridClock initialized: now 1714124932959380 us; error 29 us; skew 500 ppm
I20240426 09:48:52.961328 3448 webserver.cc:460] Webserver started at http://127.0.99.131:34345/ using document root <none> and password file <none>
I20240426 09:48:52.961866 3448 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:52.961946 3448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:52.965263 3448 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:52.967221 3462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:52.968261 3448 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:52.968364 3448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "55f9bcefe16745108099bdd94114d98f"
format_stamp: "Formatted at 2024-04-26 09:48:46 on dist-test-slave-1zqj"
I20240426 09:48:52.968744 3448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:52.986862 3448 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:52.987471 3448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:52.987644 3448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:52.988171 3448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:52.989248 3469 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240426 09:48:52.992318 3448 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 09:48:52.992377 3448 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20240426 09:48:52.992457 3448 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240426 09:48:52.993785 3448 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 09:48:52.993827 3448 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20240426 09:48:52.994050 3469 tablet_bootstrap.cc:492] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Bootstrap starting.
I20240426 09:48:53.003863 3448 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:44151
I20240426 09:48:53.003892 3576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:44151 every 8 connection(s)
I20240426 09:48:53.004906 3448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240426 09:48:53.006249 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 3448
I20240426 09:48:53.006657 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:36485
--local_ip_for_outbound_sockets=127.0.99.132
--tserver_master_addrs=127.0.99.190:33795
--webserver_port=45553
--webserver_interface=127.0.99.132
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:53.007879 3469 log.cc:824] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:53.010574 3469 tablet_bootstrap.cc:492] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:48:53.010919 3469 tablet_bootstrap.cc:492] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Bootstrap complete.
I20240426 09:48:53.012102 3469 ts_tablet_manager.cc:1367] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.006s
I20240426 09:48:53.015143 3577 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:53.015383 3577 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:53.015127 3469 raft_consensus.cc:348] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:48:53.015513 3469 raft_consensus.cc:729] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f9bcefe16745108099bdd94114d98f, State: Initialized, Role: FOLLOWER
I20240426 09:48:53.015893 3577 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:53.015959 3469 consensus_queue.cc:260] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:48:53.016115 3469 raft_consensus.cc:388] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:48:53.016178 3469 raft_consensus.cc:482] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:48:53.016286 3469 raft_consensus.cc:3037] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:48:53.017875 2256 ts_manager.cc:194] Re-registered known tserver with Master: 55f9bcefe16745108099bdd94114d98f (127.0.99.131:44151)
I20240426 09:48:53.018038 3469 raft_consensus.cc:504] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:48:53.018329 3469 leader_election.cc:302] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55f9bcefe16745108099bdd94114d98f; no voters:
I20240426 09:48:53.018586 3469 leader_election.cc:288] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [CANDIDATE]: Term 2 election: Requested vote from peers
I20240426 09:48:53.018740 3582 raft_consensus.cc:2781] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:48:53.018965 3582 raft_consensus.cc:686] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [term 2 LEADER]: Becoming Leader. State: Replica: 55f9bcefe16745108099bdd94114d98f, State: Running, Role: LEADER
I20240426 09:48:53.019014 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:41403
I20240426 09:48:53.019304 3582 consensus_queue.cc:237] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:48:53.019481 3469 ts_tablet_manager.cc:1398] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20240426 09:48:53.021337 3577 heartbeater.cc:503] Master 127.0.99.190:33795 was elected leader, sending a full tablet report...
I20240426 09:48:53.022254 2256 catalog_manager.cc:5484] T 31de0dcd76d94411ad4a0e2968960f26 P 55f9bcefe16745108099bdd94114d98f reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "55f9bcefe16745108099bdd94114d98f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } } }
W20240426 09:48:53.115669 3580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:53.115878 3580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:53.115947 3580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:53.118870 3580 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:53.118939 3580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:53.119027 3580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240426 09:48:53.122543 3580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:36485
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=45553
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.3580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:53.123437 3580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:53.124513 3580 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:53.133222 3591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:53.134217 3592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:53.134996 3594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:53.135852 3580 server_base.cc:1034] running on GCE node
I20240426 09:48:53.136339 3580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:53.136919 3580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:53.138096 3580 hybrid_clock.cc:648] HybridClock initialized: now 1714124933138072 us; error 36 us; skew 500 ppm
I20240426 09:48:53.140291 3580 webserver.cc:460] Webserver started at http://127.0.99.132:45553/ using document root <none> and password file <none>
I20240426 09:48:53.140849 3580 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:53.140961 3580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:53.144470 3580 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20240426 09:48:53.146397 3600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.147528 3580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:53.147635 3580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "a18c2b34c33241afbd517065a5b797b6"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:53.148044 3580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:53.159891 3580 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:53.160517 3580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:53.160687 3580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:53.161226 3580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:53.162178 3580 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:53.162242 3580 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.162303 3580 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:53.162338 3580 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.171700 3580 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:36485
I20240426 09:48:53.171732 3713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:36485 every 8 connection(s)
I20240426 09:48:53.172863 3580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240426 09:48:53.181180 3714 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:53.181401 3714 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:53.181831 3714 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:53.182723 2256 ts_manager.cc:194] Re-registered known tserver with Master: a18c2b34c33241afbd517065a5b797b6 (127.0.99.132:36485)
I20240426 09:48:53.183033 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 3580
I20240426 09:48:53.183466 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:45093
--local_ip_for_outbound_sockets=127.0.99.133
--tserver_master_addrs=127.0.99.190:33795
--webserver_port=37233
--webserver_interface=127.0.99.133
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:53.183471 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:40665
W20240426 09:48:53.289356 3717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:53.289564 3717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:53.289606 3717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:53.292641 3717 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:53.292707 3717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:53.292793 3717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240426 09:48:53.296336 3717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:45093
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=37233
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.3717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:53.297231 3717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:53.298336 3717 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:53.307241 3725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:53.308240 3723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:53.309078 3722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:53.309903 3717 server_base.cc:1034] running on GCE node
I20240426 09:48:53.310283 3717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:53.310873 3717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:53.312009 3717 hybrid_clock.cc:648] HybridClock initialized: now 1714124933311992 us; error 40 us; skew 500 ppm
I20240426 09:48:53.314033 3717 webserver.cc:460] Webserver started at http://127.0.99.133:37233/ using document root <none> and password file <none>
I20240426 09:48:53.314568 3717 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:53.314637 3717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:53.318032 3717 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:53.319869 3731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.320825 3717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:48:53.320923 3717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "400c68b80d144bd1b094c5cae03a111a"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:53.321314 3717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:53.341027 3717 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:53.341665 3717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:53.341818 3717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:53.342360 3717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:53.343245 3717 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:53.343290 3717 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.343336 3717 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:53.343375 3717 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.351763 3717 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:45093
I20240426 09:48:53.351799 3844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:45093 every 8 connection(s)
I20240426 09:48:53.352847 3717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240426 09:48:53.359220 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 3717
I20240426 09:48:53.359583 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:41855
--local_ip_for_outbound_sockets=127.0.99.134
--tserver_master_addrs=127.0.99.190:33795
--webserver_port=37627
--webserver_interface=127.0.99.134
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:53.361441 3845 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:53.361658 3845 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:53.362108 3845 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:53.363070 2256 ts_manager.cc:194] Re-registered known tserver with Master: 400c68b80d144bd1b094c5cae03a111a (127.0.99.133:45093)
I20240426 09:48:53.363608 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:55115
W20240426 09:48:53.467309 3848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:53.467557 3848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:53.467630 3848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:53.470642 3848 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:53.470717 3848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:53.470839 3848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240426 09:48:53.474637 3848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:41855
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=37627
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.3848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:53.475597 3848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:53.476742 3848 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:53.485631 3856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:53.486455 3848 server_base.cc:1034] running on GCE node
W20240426 09:48:53.487262 3853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:53.488052 3854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:53.488394 3848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:53.488922 3848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:53.490104 3848 hybrid_clock.cc:648] HybridClock initialized: now 1714124933490092 us; error 25 us; skew 500 ppm
I20240426 09:48:53.492128 3848 webserver.cc:460] Webserver started at http://127.0.99.134:37627/ using document root <none> and password file <none>
I20240426 09:48:53.492686 3848 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:53.492754 3848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:53.495951 3848 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240426 09:48:53.497694 3862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.498636 3848 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:53.498728 3848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "010c57b7730449c3b8c982a057e6268b"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:53.499135 3848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:53.522882 3848 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:53.523540 3848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:53.523701 3848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:53.524295 3848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:53.525347 3848 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:53.525394 3848 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.525503 3848 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:53.525545 3848 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.535354 3848 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:41855
I20240426 09:48:53.535392 3975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:41855 every 8 connection(s)
I20240426 09:48:53.536379 3848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240426 09:48:53.545203 3976 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:53.545444 3976 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:53.545882 3976 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:53.546598 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 3848
I20240426 09:48:53.546926 2256 ts_manager.cc:194] Re-registered known tserver with Master: 010c57b7730449c3b8c982a057e6268b (127.0.99.134:41855)
I20240426 09:48:53.547003 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:33487
--local_ip_for_outbound_sockets=127.0.99.135
--tserver_master_addrs=127.0.99.190:33795
--webserver_port=36559
--webserver_interface=127.0.99.135
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:48:53.547493 2256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:33021
W20240426 09:48:53.655683 3979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:48:53.655928 3979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:48:53.656002 3979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:48:53.659207 3979 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:48:53.659289 3979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:48:53.659381 3979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240426 09:48:53.663126 3979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:33487
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=36559
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.3979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:48:53.664009 3979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:48:53.665073 3979 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:48:53.673942 3987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:53.674729 3984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:48:53.675452 3985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:48:53.676254 3979 server_base.cc:1034] running on GCE node
I20240426 09:48:53.676648 3979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:48:53.677181 3979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:48:53.678357 3979 hybrid_clock.cc:648] HybridClock initialized: now 1714124933678321 us; error 44 us; skew 500 ppm
I20240426 09:48:53.680330 3979 webserver.cc:460] Webserver started at http://127.0.99.135:36559/ using document root <none> and password file <none>
I20240426 09:48:53.680956 3979 fs_manager.cc:359] Metadata directory not provided
I20240426 09:48:53.681036 3979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:48:53.684587 3979 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20240426 09:48:53.686491 3993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.687495 3979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:48:53.687600 3979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "3b591a3ffd514de7959be53199789ce8"
format_stamp: "Formatted at 2024-04-26 09:48:47 on dist-test-slave-1zqj"
I20240426 09:48:53.687987 3979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:48:53.696938 3979 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:48:53.697970 3979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:48:53.698132 3979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:48:53.698647 3979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:48:53.699509 3979 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:48:53.699558 3979 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.699617 3979 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:48:53.699642 3979 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:48:53.709343 3979 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:33487
I20240426 09:48:53.709367 4106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:33487 every 8 connection(s)
I20240426 09:48:53.710337 3979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240426 09:48:53.712819 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 3979
I20240426 09:48:53.720324 4107 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:48:53.720583 4107 heartbeater.cc:464] Registering TS with master...
I20240426 09:48:53.721158 4107 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:48:53.722225 2250 ts_manager.cc:194] Re-registered known tserver with Master: 3b591a3ffd514de7959be53199789ce8 (127.0.99.135:33487)
I20240426 09:48:53.722851 2250 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:60409
I20240426 09:48:53.909699 4035 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:53.911571 3779 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:53.912186 2509 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:48:53.912674 2378 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:48:53.912954 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:53.925012 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:53.947813 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:54.043952 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:54.048036 3779 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:54.051580 2378 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:48:54.052453 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:48:54.053936 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:54.068998 4035 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:48:54.078255 2509 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:48:54.114547 2529 consensus_queue.cc:237] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
W20240426 09:48:54.119768 2462 consensus_peers.cc:485] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 -> Peer a18c2b34c33241afbd517065a5b797b6 (127.0.99.132:36485): Couldn't send request to peer a18c2b34c33241afbd517065a5b797b6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 469ea537fdbc46d8982f23b6a1a181ab. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:48:54.185637 3714 heartbeater.cc:503] Master 127.0.99.190:33795 was elected leader, sending a full tablet report...
I20240426 09:48:54.366166 3845 heartbeater.cc:503] Master 127.0.99.190:33795 was elected leader, sending a full tablet report...
I20240426 09:48:54.549525 3976 heartbeater.cc:503] Master 127.0.99.190:33795 was elected leader, sending a full tablet report...
I20240426 09:48:54.610746 4209 ts_tablet_manager.cc:902] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: Initiating tablet copy from peer 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:48:54.611521 4209 tablet_copy_client.cc:308] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:38121
I20240426 09:48:54.615352 2549 tablet_copy_service.cc:140] P 0ad0612beac445e896fcfc0562baa296: Received BeginTabletCopySession request for tablet 469ea537fdbc46d8982f23b6a1a181ab from peer a18c2b34c33241afbd517065a5b797b6 ({username='slave'} at 127.0.99.132:46581)
I20240426 09:48:54.615481 2549 tablet_copy_service.cc:161] P 0ad0612beac445e896fcfc0562baa296: Beginning new tablet copy session on tablet 469ea537fdbc46d8982f23b6a1a181ab from peer a18c2b34c33241afbd517065a5b797b6 at {username='slave'} at 127.0.99.132:46581: session id = a18c2b34c33241afbd517065a5b797b6-469ea537fdbc46d8982f23b6a1a181ab
I20240426 09:48:54.616597 2549 tablet_copy_source_session.cc:205] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:48:54.619719 4209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 469ea537fdbc46d8982f23b6a1a181ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:54.624600 4209 tablet_copy_client.cc:783] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: tablet copy: Starting download of 0 data blocks...
I20240426 09:48:54.624792 4209 tablet_copy_client.cc:647] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: tablet copy: Starting download of 1 WAL segments...
I20240426 09:48:54.626394 4209 tablet_copy_client.cc:523] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:48:54.627862 4209 tablet_bootstrap.cc:492] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: Bootstrap starting.
I20240426 09:48:54.633138 4209 log.cc:824] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:54.651091 4209 tablet_bootstrap.cc:492] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:48:54.651513 4209 tablet_bootstrap.cc:492] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: Bootstrap complete.
I20240426 09:48:54.651806 4209 ts_tablet_manager.cc:1367] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: Time spent bootstrapping tablet: real 0.024s user 0.023s sys 0.001s
I20240426 09:48:54.653805 4209 raft_consensus.cc:348] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:54.654356 4209 raft_consensus.cc:729] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a18c2b34c33241afbd517065a5b797b6, State: Initialized, Role: FOLLOWER
I20240426 09:48:54.654628 4209 consensus_queue.cc:260] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:48:54.655450 4209 ts_tablet_manager.cc:1398] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:48:54.656270 2549 tablet_copy_service.cc:342] P 0ad0612beac445e896fcfc0562baa296: Request end of tablet copy session a18c2b34c33241afbd517065a5b797b6-469ea537fdbc46d8982f23b6a1a181ab received from {username='slave'} at 127.0.99.132:46581
I20240426 09:48:54.656374 2549 tablet_copy_service.cc:433] P 0ad0612beac445e896fcfc0562baa296: ending tablet copy session a18c2b34c33241afbd517065a5b797b6-469ea537fdbc46d8982f23b6a1a181ab on tablet 469ea537fdbc46d8982f23b6a1a181ab with peer a18c2b34c33241afbd517065a5b797b6
I20240426 09:48:54.725389 4107 heartbeater.cc:503] Master 127.0.99.190:33795 was elected leader, sending a full tablet report...
I20240426 09:48:55.158167 3657 raft_consensus.cc:1203] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:48:55.159616 4219 raft_consensus.cc:2932] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a18c2b34c33241afbd517065a5b797b6 (127.0.99.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:48:55.160121 3657 raft_consensus.cc:2932] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a18c2b34c33241afbd517065a5b797b6 (127.0.99.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:48:55.161362 4219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35176 (request call id 0) took 1047 ms. Trace:
I20240426 09:48:55.161496 4219 rpcz_store.cc:276] 0426 09:48:54.113922 (+ 0us) service_pool.cc:168] Inserting onto call queue
0426 09:48:54.113981 (+ 59us) service_pool.cc:225] Handling call
0426 09:48:54.115381 (+ 1400us) log.cc:842] Serialized 257 byte log entry
0426 09:48:55.161344 (+1045963us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":47,"raft.queue_time_us":120,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":251,"spinlock_wait_cycles":3328,"thread_start_us":96,"threads_started":2,"wal-append.queue_time_us":130,"wal-append.run_cpu_time_us":303,"wal-append.run_wall_time_us":1000364}
I20240426 09:48:55.162813 2250 catalog_manager.cc:5484] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 reported cstate change: config changed from index -1 to 4, VOTER a18c2b34c33241afbd517065a5b797b6 (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:48:55.182561 2398 consensus_queue.cc:237] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
W20240426 09:48:55.187592 2332 consensus_peers.cc:485] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d -> Peer 010c57b7730449c3b8c982a057e6268b (127.0.99.134:41855): Couldn't send request to peer 010c57b7730449c3b8c982a057e6268b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4810d8df46c4404875a4a53a50ecb60. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:48:55.770079 4248 ts_tablet_manager.cc:902] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: Initiating tablet copy from peer 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:48:55.770809 4248 tablet_copy_client.cc:308] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:34009
I20240426 09:48:55.774828 2418 tablet_copy_service.cc:140] P 506a29f8a6094b909536aa04d980679d: Received BeginTabletCopySession request for tablet c4810d8df46c4404875a4a53a50ecb60 from peer 010c57b7730449c3b8c982a057e6268b ({username='slave'} at 127.0.99.134:47717)
I20240426 09:48:55.774986 2418 tablet_copy_service.cc:161] P 506a29f8a6094b909536aa04d980679d: Beginning new tablet copy session on tablet c4810d8df46c4404875a4a53a50ecb60 from peer 010c57b7730449c3b8c982a057e6268b at {username='slave'} at 127.0.99.134:47717: session id = 010c57b7730449c3b8c982a057e6268b-c4810d8df46c4404875a4a53a50ecb60
I20240426 09:48:55.776088 2418 tablet_copy_source_session.cc:205] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:48:55.779474 4248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4810d8df46c4404875a4a53a50ecb60. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:55.784988 4248 tablet_copy_client.cc:783] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: tablet copy: Starting download of 0 data blocks...
I20240426 09:48:55.785171 4248 tablet_copy_client.cc:647] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: tablet copy: Starting download of 1 WAL segments...
I20240426 09:48:55.786844 4248 tablet_copy_client.cc:523] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:48:55.788239 4248 tablet_bootstrap.cc:492] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: Bootstrap starting.
I20240426 09:48:55.793536 4248 log.cc:824] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:55.813323 4248 tablet_bootstrap.cc:492] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:48:55.813676 4248 tablet_bootstrap.cc:492] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: Bootstrap complete.
I20240426 09:48:55.813930 4248 ts_tablet_manager.cc:1367] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.000s
I20240426 09:48:55.815819 4248 raft_consensus.cc:348] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:55.816321 4248 raft_consensus.cc:729] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010c57b7730449c3b8c982a057e6268b, State: Initialized, Role: FOLLOWER
I20240426 09:48:55.816568 4248 consensus_queue.cc:260] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:48:55.817288 4248 ts_tablet_manager.cc:1398] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20240426 09:48:55.818097 2418 tablet_copy_service.cc:342] P 506a29f8a6094b909536aa04d980679d: Request end of tablet copy session 010c57b7730449c3b8c982a057e6268b-c4810d8df46c4404875a4a53a50ecb60 received from {username='slave'} at 127.0.99.134:47717
I20240426 09:48:55.818185 2418 tablet_copy_service.cc:433] P 506a29f8a6094b909536aa04d980679d: ending tablet copy session 010c57b7730449c3b8c982a057e6268b-c4810d8df46c4404875a4a53a50ecb60 on tablet c4810d8df46c4404875a4a53a50ecb60 with peer 010c57b7730449c3b8c982a057e6268b
I20240426 09:48:56.323439 3930 raft_consensus.cc:1203] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:48:56.324752 4259 raft_consensus.cc:2932] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 010c57b7730449c3b8c982a057e6268b (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:48:56.325373 3930 raft_consensus.cc:2932] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 010c57b7730449c3b8c982a057e6268b (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:48:56.326661 4259 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44742 (request call id 0) took 1144 ms. Trace:
I20240426 09:48:56.326815 4259 rpcz_store.cc:276] 0426 09:48:55.181936 (+ 0us) service_pool.cc:168] Inserting onto call queue
0426 09:48:55.181986 (+ 50us) service_pool.cc:225] Handling call
0426 09:48:55.183377 (+ 1391us) log.cc:842] Serialized 257 byte log entry
0426 09:48:56.326584 (+1143207us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":49,"raft.queue_time_us":106,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":241,"spinlock_wait_cycles":18176,"thread_start_us":115,"threads_started":2,"wal-append.queue_time_us":146,"wal-append.run_cpu_time_us":315,"wal-append.run_wall_time_us":1000384}
I20240426 09:48:56.328171 2250 catalog_manager.cc:5484] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b reported cstate change: config changed from index -1 to 4, VOTER 010c57b7730449c3b8c982a057e6268b (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:48:56.345139 2529 consensus_queue.cc:237] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
W20240426 09:48:56.349761 2463 consensus_peers.cc:485] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 -> Peer 3b591a3ffd514de7959be53199789ce8 (127.0.99.135:33487): Couldn't send request to peer 3b591a3ffd514de7959be53199789ce8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 388e1882272944a3893813caef6cd903. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:48:56.766566 4286 ts_tablet_manager.cc:902] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: Initiating tablet copy from peer 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:48:56.767326 4286 tablet_copy_client.cc:308] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:38121
I20240426 09:48:56.770970 2549 tablet_copy_service.cc:140] P 0ad0612beac445e896fcfc0562baa296: Received BeginTabletCopySession request for tablet 388e1882272944a3893813caef6cd903 from peer 3b591a3ffd514de7959be53199789ce8 ({username='slave'} at 127.0.99.135:47777)
I20240426 09:48:56.771072 2549 tablet_copy_service.cc:161] P 0ad0612beac445e896fcfc0562baa296: Beginning new tablet copy session on tablet 388e1882272944a3893813caef6cd903 from peer 3b591a3ffd514de7959be53199789ce8 at {username='slave'} at 127.0.99.135:47777: session id = 3b591a3ffd514de7959be53199789ce8-388e1882272944a3893813caef6cd903
I20240426 09:48:56.771862 2549 tablet_copy_source_session.cc:205] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:48:56.775247 4286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 388e1882272944a3893813caef6cd903. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:56.780612 4286 tablet_copy_client.cc:783] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: tablet copy: Starting download of 0 data blocks...
I20240426 09:48:56.780849 4286 tablet_copy_client.cc:647] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: tablet copy: Starting download of 1 WAL segments...
I20240426 09:48:56.782388 4286 tablet_copy_client.cc:523] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:48:56.783871 4286 tablet_bootstrap.cc:492] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: Bootstrap starting.
I20240426 09:48:56.788749 4286 log.cc:824] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:56.807518 4286 tablet_bootstrap.cc:492] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:48:56.807904 4286 tablet_bootstrap.cc:492] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: Bootstrap complete.
I20240426 09:48:56.808177 4286 ts_tablet_manager.cc:1367] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: Time spent bootstrapping tablet: real 0.024s user 0.024s sys 0.000s
I20240426 09:48:56.810066 4286 raft_consensus.cc:348] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:56.810573 4286 raft_consensus.cc:729] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b591a3ffd514de7959be53199789ce8, State: Initialized, Role: FOLLOWER
I20240426 09:48:56.810883 4286 consensus_queue.cc:260] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:48:56.811668 4286 ts_tablet_manager.cc:1398] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20240426 09:48:56.812433 2549 tablet_copy_service.cc:342] P 0ad0612beac445e896fcfc0562baa296: Request end of tablet copy session 3b591a3ffd514de7959be53199789ce8-388e1882272944a3893813caef6cd903 received from {username='slave'} at 127.0.99.135:47777
I20240426 09:48:56.812541 2549 tablet_copy_service.cc:433] P 0ad0612beac445e896fcfc0562baa296: ending tablet copy session 3b591a3ffd514de7959be53199789ce8-388e1882272944a3893813caef6cd903 on tablet 388e1882272944a3893813caef6cd903 with peer 3b591a3ffd514de7959be53199789ce8
I20240426 09:48:57.365727 4061 raft_consensus.cc:1203] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:48:57.367072 4295 raft_consensus.cc:2932] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b591a3ffd514de7959be53199789ce8 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:48:57.367713 4061 raft_consensus.cc:2932] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b591a3ffd514de7959be53199789ce8 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:48:57.368682 4295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35186 (request call id 0) took 1023 ms. Trace:
I20240426 09:48:57.368750 4295 rpcz_store.cc:276] 0426 09:48:56.344803 (+ 0us) service_pool.cc:168] Inserting onto call queue
0426 09:48:56.344860 (+ 57us) service_pool.cc:225] Handling call
0426 09:48:56.345586 (+ 726us) log.cc:842] Serialized 257 byte log entry
0426 09:48:57.368663 (+1023077us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":40,"raft.queue_time_us":44,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":65,"threads_started":1,"wal-append.queue_time_us":130,"wal-append.run_cpu_time_us":306,"wal-append.run_wall_time_us":1000390}
I20240426 09:48:57.370169 2252 catalog_manager.cc:5484] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 reported cstate change: config changed from index -1 to 4, VOTER 3b591a3ffd514de7959be53199789ce8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:48:57.403000 2398 consensus_queue.cc:237] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
W20240426 09:48:57.407548 2333 consensus_peers.cc:485] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d -> Peer 400c68b80d144bd1b094c5cae03a111a (127.0.99.133:45093): Couldn't send request to peer 400c68b80d144bd1b094c5cae03a111a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: faeca7ce592f4ffe806903cb915c75d4. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:48:57.891109 4322 ts_tablet_manager.cc:902] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: Initiating tablet copy from peer 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:48:57.891871 4322 tablet_copy_client.cc:308] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:34009
I20240426 09:48:57.895455 2418 tablet_copy_service.cc:140] P 506a29f8a6094b909536aa04d980679d: Received BeginTabletCopySession request for tablet faeca7ce592f4ffe806903cb915c75d4 from peer 400c68b80d144bd1b094c5cae03a111a ({username='slave'} at 127.0.99.133:51457)
I20240426 09:48:57.895560 2418 tablet_copy_service.cc:161] P 506a29f8a6094b909536aa04d980679d: Beginning new tablet copy session on tablet faeca7ce592f4ffe806903cb915c75d4 from peer 400c68b80d144bd1b094c5cae03a111a at {username='slave'} at 127.0.99.133:51457: session id = 400c68b80d144bd1b094c5cae03a111a-faeca7ce592f4ffe806903cb915c75d4
I20240426 09:48:57.896304 2418 tablet_copy_source_session.cc:205] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:48:57.899360 4322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faeca7ce592f4ffe806903cb915c75d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:57.905021 4322 tablet_copy_client.cc:783] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: tablet copy: Starting download of 0 data blocks...
I20240426 09:48:57.905227 4322 tablet_copy_client.cc:647] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: tablet copy: Starting download of 1 WAL segments...
I20240426 09:48:57.906713 4322 tablet_copy_client.cc:523] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:48:57.908205 4322 tablet_bootstrap.cc:492] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: Bootstrap starting.
I20240426 09:48:57.913142 4322 log.cc:824] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: Log is configured to *not* fsync() on all Append() calls
I20240426 09:48:57.933444 4322 tablet_bootstrap.cc:492] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:48:57.933867 4322 tablet_bootstrap.cc:492] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: Bootstrap complete.
I20240426 09:48:57.934175 4322 ts_tablet_manager.cc:1367] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.000s
I20240426 09:48:57.936121 4322 raft_consensus.cc:348] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:57.936820 4322 raft_consensus.cc:729] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 400c68b80d144bd1b094c5cae03a111a, State: Initialized, Role: FOLLOWER
I20240426 09:48:57.937134 4322 consensus_queue.cc:260] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:48:57.937968 4322 ts_tablet_manager.cc:1398] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20240426 09:48:57.938794 2418 tablet_copy_service.cc:342] P 506a29f8a6094b909536aa04d980679d: Request end of tablet copy session 400c68b80d144bd1b094c5cae03a111a-faeca7ce592f4ffe806903cb915c75d4 received from {username='slave'} at 127.0.99.133:51457
I20240426 09:48:57.938894 2418 tablet_copy_service.cc:433] P 506a29f8a6094b909536aa04d980679d: ending tablet copy session 400c68b80d144bd1b094c5cae03a111a-faeca7ce592f4ffe806903cb915c75d4 on tablet faeca7ce592f4ffe806903cb915c75d4 with peer 400c68b80d144bd1b094c5cae03a111a
I20240426 09:48:58.309489 3799 raft_consensus.cc:1203] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:48:58.310745 4330 raft_consensus.cc:2932] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 400c68b80d144bd1b094c5cae03a111a (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:48:58.311345 3799 raft_consensus.cc:2932] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 400c68b80d144bd1b094c5cae03a111a (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:48:58.314174 2250 catalog_manager.cc:5484] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d reported cstate change: config changed from index -1 to 4, VOTER 400c68b80d144bd1b094c5cae03a111a (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:48:58.330350 2398 consensus_queue.cc:237] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
W20240426 09:48:58.335098 2331 consensus_peers.cc:485] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d -> Peer 55f9bcefe16745108099bdd94114d98f (127.0.99.131:44151): Couldn't send request to peer 55f9bcefe16745108099bdd94114d98f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0db2ff8ccef842a2ac866f85b0e05134. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:48:58.740891 4355 ts_tablet_manager.cc:902] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: Initiating tablet copy from peer 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:48:58.741580 4355 tablet_copy_client.cc:308] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:34009
I20240426 09:48:58.745388 2418 tablet_copy_service.cc:140] P 506a29f8a6094b909536aa04d980679d: Received BeginTabletCopySession request for tablet 0db2ff8ccef842a2ac866f85b0e05134 from peer 55f9bcefe16745108099bdd94114d98f ({username='slave'} at 127.0.99.131:41599)
I20240426 09:48:58.745523 2418 tablet_copy_service.cc:161] P 506a29f8a6094b909536aa04d980679d: Beginning new tablet copy session on tablet 0db2ff8ccef842a2ac866f85b0e05134 from peer 55f9bcefe16745108099bdd94114d98f at {username='slave'} at 127.0.99.131:41599: session id = 55f9bcefe16745108099bdd94114d98f-0db2ff8ccef842a2ac866f85b0e05134
I20240426 09:48:58.746506 2418 tablet_copy_source_session.cc:205] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:48:58.748236 4355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db2ff8ccef842a2ac866f85b0e05134. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:58.751866 4355 tablet_copy_client.cc:783] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: tablet copy: Starting download of 0 data blocks...
I20240426 09:48:58.752171 4355 tablet_copy_client.cc:647] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: tablet copy: Starting download of 1 WAL segments...
I20240426 09:48:58.753829 4355 tablet_copy_client.cc:523] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:48:58.755517 4355 tablet_bootstrap.cc:492] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: Bootstrap starting.
I20240426 09:48:58.795706 4355 tablet_bootstrap.cc:492] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:48:58.796092 4355 tablet_bootstrap.cc:492] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: Bootstrap complete.
I20240426 09:48:58.796279 4355 ts_tablet_manager.cc:1367] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: Time spent bootstrapping tablet: real 0.041s user 0.039s sys 0.000s
I20240426 09:48:58.796828 4355 raft_consensus.cc:348] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:48:58.797166 4355 raft_consensus.cc:729] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f9bcefe16745108099bdd94114d98f, State: Initialized, Role: FOLLOWER
I20240426 09:48:58.797261 4355 consensus_queue.cc:260] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:48:58.797565 4355 ts_tablet_manager.cc:1398] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240426 09:48:58.798275 2418 tablet_copy_service.cc:342] P 506a29f8a6094b909536aa04d980679d: Request end of tablet copy session 55f9bcefe16745108099bdd94114d98f-0db2ff8ccef842a2ac866f85b0e05134 received from {username='slave'} at 127.0.99.131:41599
I20240426 09:48:58.798353 2418 tablet_copy_service.cc:433] P 506a29f8a6094b909536aa04d980679d: ending tablet copy session 55f9bcefe16745108099bdd94114d98f-0db2ff8ccef842a2ac866f85b0e05134 on tablet 0db2ff8ccef842a2ac866f85b0e05134 with peer 55f9bcefe16745108099bdd94114d98f
I20240426 09:48:59.219321 3531 raft_consensus.cc:1203] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:48:59.220384 4363 raft_consensus.cc:2932] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 55f9bcefe16745108099bdd94114d98f (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } } }
I20240426 09:48:59.220865 3531 raft_consensus.cc:2932] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 55f9bcefe16745108099bdd94114d98f (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } } }
I20240426 09:48:59.223484 2250 catalog_manager.cc:5484] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d reported cstate change: config changed from index -1 to 4, VOTER 55f9bcefe16745108099bdd94114d98f (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } } }
I20240426 09:48:59.240161 2529 consensus_queue.cc:237] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
W20240426 09:48:59.241576 2462 consensus_peers.cc:485] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 -> Peer a18c2b34c33241afbd517065a5b797b6 (127.0.99.132:36485): Couldn't send request to peer a18c2b34c33241afbd517065a5b797b6. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43b0427d465142a1b7639dd269112b15. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:48:59.788837 4386 ts_tablet_manager.cc:902] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: Initiating tablet copy from peer 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:48:59.789337 4386 tablet_copy_client.cc:308] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:38121
I20240426 09:48:59.789887 2549 tablet_copy_service.cc:140] P 0ad0612beac445e896fcfc0562baa296: Received BeginTabletCopySession request for tablet 43b0427d465142a1b7639dd269112b15 from peer a18c2b34c33241afbd517065a5b797b6 ({username='slave'} at 127.0.99.132:46581)
I20240426 09:48:59.790001 2549 tablet_copy_service.cc:161] P 0ad0612beac445e896fcfc0562baa296: Beginning new tablet copy session on tablet 43b0427d465142a1b7639dd269112b15 from peer a18c2b34c33241afbd517065a5b797b6 at {username='slave'} at 127.0.99.132:46581: session id = a18c2b34c33241afbd517065a5b797b6-43b0427d465142a1b7639dd269112b15
I20240426 09:48:59.790836 2549 tablet_copy_source_session.cc:205] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:48:59.791703 4386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43b0427d465142a1b7639dd269112b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:48:59.793453 4386 tablet_copy_client.cc:783] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: tablet copy: Starting download of 0 data blocks...
I20240426 09:48:59.793597 4386 tablet_copy_client.cc:647] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: tablet copy: Starting download of 1 WAL segments...
I20240426 09:48:59.794752 4386 tablet_copy_client.cc:523] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:48:59.795744 4386 tablet_bootstrap.cc:492] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: Bootstrap starting.
I20240426 09:48:59.811667 4386 tablet_bootstrap.cc:492] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:48:59.811975 4386 tablet_bootstrap.cc:492] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: Bootstrap complete.
I20240426 09:48:59.812141 4386 ts_tablet_manager.cc:1367] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: Time spent bootstrapping tablet: real 0.016s user 0.016s sys 0.000s
I20240426 09:48:59.812669 4386 raft_consensus.cc:348] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:48:59.812891 4386 raft_consensus.cc:729] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a18c2b34c33241afbd517065a5b797b6, State: Initialized, Role: FOLLOWER
I20240426 09:48:59.812985 4386 consensus_queue.cc:260] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:48:59.813333 4386 ts_tablet_manager.cc:1398] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:48:59.813956 2549 tablet_copy_service.cc:342] P 0ad0612beac445e896fcfc0562baa296: Request end of tablet copy session a18c2b34c33241afbd517065a5b797b6-43b0427d465142a1b7639dd269112b15 received from {username='slave'} at 127.0.99.132:46581
I20240426 09:48:59.814050 2549 tablet_copy_service.cc:433] P 0ad0612beac445e896fcfc0562baa296: ending tablet copy session a18c2b34c33241afbd517065a5b797b6-43b0427d465142a1b7639dd269112b15 on tablet 43b0427d465142a1b7639dd269112b15 with peer a18c2b34c33241afbd517065a5b797b6
I20240426 09:49:00.240906 3657 raft_consensus.cc:1203] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:00.241765 4393 raft_consensus.cc:2932] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a18c2b34c33241afbd517065a5b797b6 (127.0.99.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:49:00.242214 3657 raft_consensus.cc:2932] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a18c2b34c33241afbd517065a5b797b6 (127.0.99.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:49:00.243371 4393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35204 (request call id 0) took 1003 ms. Trace:
I20240426 09:49:00.243479 4393 rpcz_store.cc:276] 0426 09:48:59.239796 (+ 0us) service_pool.cc:168] Inserting onto call queue
0426 09:48:59.239867 (+ 71us) service_pool.cc:225] Handling call
0426 09:48:59.240560 (+ 693us) log.cc:842] Serialized 257 byte log entry
0426 09:49:00.243351 (+1002791us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":38,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":140,"spinlock_wait_cycles":1792,"thread_start_us":69,"threads_started":1,"wal-append.queue_time_us":142,"wal-append.run_cpu_time_us":323,"wal-append.run_wall_time_us":1000419}
I20240426 09:49:00.244516 2252 catalog_manager.cc:5484] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 reported cstate change: config changed from index -1 to 4, VOTER a18c2b34c33241afbd517065a5b797b6 (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:49:00.261643 2398 consensus_queue.cc:237] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
W20240426 09:49:00.262933 2332 consensus_peers.cc:485] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d -> Peer 010c57b7730449c3b8c982a057e6268b (127.0.99.134:41855): Couldn't send request to peer 010c57b7730449c3b8c982a057e6268b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 54dd16182f294fe9af3ff82cd57b7e5a. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:00.755754 4418 ts_tablet_manager.cc:902] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: Initiating tablet copy from peer 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:49:00.756246 4418 tablet_copy_client.cc:308] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:34009
I20240426 09:49:00.756742 2418 tablet_copy_service.cc:140] P 506a29f8a6094b909536aa04d980679d: Received BeginTabletCopySession request for tablet 54dd16182f294fe9af3ff82cd57b7e5a from peer 010c57b7730449c3b8c982a057e6268b ({username='slave'} at 127.0.99.134:47717)
I20240426 09:49:00.756855 2418 tablet_copy_service.cc:161] P 506a29f8a6094b909536aa04d980679d: Beginning new tablet copy session on tablet 54dd16182f294fe9af3ff82cd57b7e5a from peer 010c57b7730449c3b8c982a057e6268b at {username='slave'} at 127.0.99.134:47717: session id = 010c57b7730449c3b8c982a057e6268b-54dd16182f294fe9af3ff82cd57b7e5a
I20240426 09:49:00.757681 2418 tablet_copy_source_session.cc:205] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:00.758582 4418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54dd16182f294fe9af3ff82cd57b7e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:00.760330 4418 tablet_copy_client.cc:783] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:00.760470 4418 tablet_copy_client.cc:647] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:00.761583 4418 tablet_copy_client.cc:523] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:00.762653 4418 tablet_bootstrap.cc:492] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: Bootstrap starting.
I20240426 09:49:00.780732 4418 tablet_bootstrap.cc:492] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:49:00.781025 4418 tablet_bootstrap.cc:492] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: Bootstrap complete.
I20240426 09:49:00.781196 4418 ts_tablet_manager.cc:1367] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: Time spent bootstrapping tablet: real 0.019s user 0.017s sys 0.000s
I20240426 09:49:00.781718 4418 raft_consensus.cc:348] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:49:00.781941 4418 raft_consensus.cc:729] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010c57b7730449c3b8c982a057e6268b, State: Initialized, Role: FOLLOWER
I20240426 09:49:00.782042 4418 consensus_queue.cc:260] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:49:00.782351 4418 ts_tablet_manager.cc:1398] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240426 09:49:00.783001 2418 tablet_copy_service.cc:342] P 506a29f8a6094b909536aa04d980679d: Request end of tablet copy session 010c57b7730449c3b8c982a057e6268b-54dd16182f294fe9af3ff82cd57b7e5a received from {username='slave'} at 127.0.99.134:47717
I20240426 09:49:00.783087 2418 tablet_copy_service.cc:433] P 506a29f8a6094b909536aa04d980679d: ending tablet copy session 010c57b7730449c3b8c982a057e6268b-54dd16182f294fe9af3ff82cd57b7e5a on tablet 54dd16182f294fe9af3ff82cd57b7e5a with peer 010c57b7730449c3b8c982a057e6268b
I20240426 09:49:01.280216 3930 raft_consensus.cc:1203] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:01.281138 4425 raft_consensus.cc:2932] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 010c57b7730449c3b8c982a057e6268b (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:49:01.281533 3930 raft_consensus.cc:2932] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 010c57b7730449c3b8c982a057e6268b (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:49:01.282591 4425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:44794 (request call id 0) took 1021 ms. Trace:
I20240426 09:49:01.282707 4425 rpcz_store.cc:276] 0426 09:49:00.261321 (+ 0us) service_pool.cc:168] Inserting onto call queue
0426 09:49:00.261372 (+ 51us) service_pool.cc:225] Handling call
0426 09:49:00.262012 (+ 640us) log.cc:842] Serialized 257 byte log entry
0426 09:49:01.282575 (+1020563us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":34,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":135,"thread_start_us":64,"threads_started":1,"wal-append.queue_time_us":125,"wal-append.run_cpu_time_us":320,"wal-append.run_wall_time_us":1000394}
I20240426 09:49:01.284547 2252 catalog_manager.cc:5484] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d reported cstate change: config changed from index -1 to 4, VOTER 010c57b7730449c3b8c982a057e6268b (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:49:01.300755 2529 consensus_queue.cc:237] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
W20240426 09:49:01.302031 2463 consensus_peers.cc:485] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 -> Peer 3b591a3ffd514de7959be53199789ce8 (127.0.99.135:33487): Couldn't send request to peer 3b591a3ffd514de7959be53199789ce8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7de16751060b4b91b679ba22803ae980. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:01.891862 4450 ts_tablet_manager.cc:902] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: Initiating tablet copy from peer 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:49:01.892446 4450 tablet_copy_client.cc:308] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:38121
I20240426 09:49:01.892977 2549 tablet_copy_service.cc:140] P 0ad0612beac445e896fcfc0562baa296: Received BeginTabletCopySession request for tablet 7de16751060b4b91b679ba22803ae980 from peer 3b591a3ffd514de7959be53199789ce8 ({username='slave'} at 127.0.99.135:47777)
I20240426 09:49:01.893086 2549 tablet_copy_service.cc:161] P 0ad0612beac445e896fcfc0562baa296: Beginning new tablet copy session on tablet 7de16751060b4b91b679ba22803ae980 from peer 3b591a3ffd514de7959be53199789ce8 at {username='slave'} at 127.0.99.135:47777: session id = 3b591a3ffd514de7959be53199789ce8-7de16751060b4b91b679ba22803ae980
I20240426 09:49:01.893791 2549 tablet_copy_source_session.cc:205] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:01.894615 4450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7de16751060b4b91b679ba22803ae980. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:01.896500 4450 tablet_copy_client.cc:783] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:01.896699 4450 tablet_copy_client.cc:647] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:01.897960 4450 tablet_copy_client.cc:523] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:01.899086 4450 tablet_bootstrap.cc:492] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: Bootstrap starting.
I20240426 09:49:01.916760 4450 tablet_bootstrap.cc:492] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:49:01.917068 4450 tablet_bootstrap.cc:492] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: Bootstrap complete.
I20240426 09:49:01.917263 4450 ts_tablet_manager.cc:1367] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
I20240426 09:49:01.917776 4450 raft_consensus.cc:348] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:49:01.918004 4450 raft_consensus.cc:729] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b591a3ffd514de7959be53199789ce8, State: Initialized, Role: FOLLOWER
I20240426 09:49:01.918102 4450 consensus_queue.cc:260] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:49:01.918473 4450 ts_tablet_manager.cc:1398] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240426 09:49:01.919083 2549 tablet_copy_service.cc:342] P 0ad0612beac445e896fcfc0562baa296: Request end of tablet copy session 3b591a3ffd514de7959be53199789ce8-7de16751060b4b91b679ba22803ae980 received from {username='slave'} at 127.0.99.135:47777
I20240426 09:49:01.919165 2549 tablet_copy_service.cc:433] P 0ad0612beac445e896fcfc0562baa296: ending tablet copy session 3b591a3ffd514de7959be53199789ce8-7de16751060b4b91b679ba22803ae980 on tablet 7de16751060b4b91b679ba22803ae980 with peer 3b591a3ffd514de7959be53199789ce8
I20240426 09:49:02.456696 4061 raft_consensus.cc:1203] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:02.457676 4458 raft_consensus.cc:2932] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b591a3ffd514de7959be53199789ce8 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:49:02.458096 4061 raft_consensus.cc:2932] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b591a3ffd514de7959be53199789ce8 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:49:02.459216 4458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:35220 (request call id 0) took 1158 ms. Trace:
I20240426 09:49:02.459329 4458 rpcz_store.cc:276] 0426 09:49:01.300409 (+ 0us) service_pool.cc:168] Inserting onto call queue
0426 09:49:01.300469 (+ 60us) service_pool.cc:225] Handling call
0426 09:49:01.301118 (+ 649us) log.cc:842] Serialized 257 byte log entry
0426 09:49:02.459201 (+1158083us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":35,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":164,"spinlock_wait_cycles":3328,"thread_start_us":64,"threads_started":1,"wal-append.queue_time_us":134,"wal-append.run_cpu_time_us":316,"wal-append.run_wall_time_us":1000387}
I20240426 09:49:02.460654 2256 catalog_manager.cc:5484] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 reported cstate change: config changed from index -1 to 4, VOTER 3b591a3ffd514de7959be53199789ce8 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:49:02.482704 2398 consensus_queue.cc:237] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
W20240426 09:49:02.484232 2333 consensus_peers.cc:485] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d -> Peer 400c68b80d144bd1b094c5cae03a111a (127.0.99.133:45093): Couldn't send request to peer 400c68b80d144bd1b094c5cae03a111a. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87ca4c907ced49ecb4b8acc48efe734d. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:03.015408 4483 ts_tablet_manager.cc:902] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: Initiating tablet copy from peer 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:49:03.015908 4483 tablet_copy_client.cc:308] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:34009
I20240426 09:49:03.016417 2418 tablet_copy_service.cc:140] P 506a29f8a6094b909536aa04d980679d: Received BeginTabletCopySession request for tablet 87ca4c907ced49ecb4b8acc48efe734d from peer 400c68b80d144bd1b094c5cae03a111a ({username='slave'} at 127.0.99.133:51457)
I20240426 09:49:03.016521 2418 tablet_copy_service.cc:161] P 506a29f8a6094b909536aa04d980679d: Beginning new tablet copy session on tablet 87ca4c907ced49ecb4b8acc48efe734d from peer 400c68b80d144bd1b094c5cae03a111a at {username='slave'} at 127.0.99.133:51457: session id = 400c68b80d144bd1b094c5cae03a111a-87ca4c907ced49ecb4b8acc48efe734d
I20240426 09:49:03.017187 2418 tablet_copy_source_session.cc:205] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:03.018034 4483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87ca4c907ced49ecb4b8acc48efe734d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:03.019853 4483 tablet_copy_client.cc:783] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:03.020063 4483 tablet_copy_client.cc:647] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:03.021086 4483 tablet_copy_client.cc:523] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:03.022104 4483 tablet_bootstrap.cc:492] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: Bootstrap starting.
I20240426 09:49:03.039072 4483 tablet_bootstrap.cc:492] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:49:03.039381 4483 tablet_bootstrap.cc:492] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: Bootstrap complete.
I20240426 09:49:03.039587 4483 ts_tablet_manager.cc:1367] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.004s
I20240426 09:49:03.040148 4483 raft_consensus.cc:348] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:49:03.040413 4483 raft_consensus.cc:729] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 400c68b80d144bd1b094c5cae03a111a, State: Initialized, Role: FOLLOWER
I20240426 09:49:03.040522 4483 consensus_queue.cc:260] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:49:03.040920 4483 ts_tablet_manager.cc:1398] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240426 09:49:03.041548 2418 tablet_copy_service.cc:342] P 506a29f8a6094b909536aa04d980679d: Request end of tablet copy session 400c68b80d144bd1b094c5cae03a111a-87ca4c907ced49ecb4b8acc48efe734d received from {username='slave'} at 127.0.99.133:51457
I20240426 09:49:03.041651 2418 tablet_copy_service.cc:433] P 506a29f8a6094b909536aa04d980679d: ending tablet copy session 400c68b80d144bd1b094c5cae03a111a-87ca4c907ced49ecb4b8acc48efe734d on tablet 87ca4c907ced49ecb4b8acc48efe734d with peer 400c68b80d144bd1b094c5cae03a111a
I20240426 09:49:03.428341 3799 raft_consensus.cc:1203] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:03.429230 4490 raft_consensus.cc:2932] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 400c68b80d144bd1b094c5cae03a111a (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:49:03.429673 3799 raft_consensus.cc:2932] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 400c68b80d144bd1b094c5cae03a111a (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:49:03.432420 2256 catalog_manager.cc:5484] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a reported cstate change: config changed from index -1 to 4, VOTER 400c68b80d144bd1b094c5cae03a111a (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:49:03.490106 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:03.490418 2509 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:03.491874 3779 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.491911 2378 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:03.493636 4035 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.507347 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.507354 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.548007 2398 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "87ca4c907ced49ecb4b8acc48efe734d"
dest_uuid: "506a29f8a6094b909536aa04d980679d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60876
I20240426 09:49:03.548177 2398 raft_consensus.cc:595] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:03.597066 2509 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:03.599375 4035 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.600246 2378 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:03.603942 3779 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.606945 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:03.607721 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.611949 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.698204 4035 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.698295 2509 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:03.703112 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.704218 3779 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.709424 2378 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:03.719940 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:03.730430 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.752291 2529 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7de16751060b4b91b679ba22803ae980"
dest_uuid: "0ad0612beac445e896fcfc0562baa296"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:57306
I20240426 09:49:03.752558 2529 raft_consensus.cc:595] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:03.805353 2509 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:03.806983 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.809417 2378 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:03.810329 3779 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.824790 4035 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.825776 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.828047 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:03.885397 4667 raft_consensus.cc:979] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: : Instructing follower 3b591a3ffd514de7959be53199789ce8 to start an election
I20240426 09:49:03.885514 4667 raft_consensus.cc:1067] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Signalling peer 3b591a3ffd514de7959be53199789ce8 to start an election
I20240426 09:49:03.886090 4061 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7de16751060b4b91b679ba22803ae980"
dest_uuid: "3b591a3ffd514de7959be53199789ce8"
from {username='slave'} at 127.0.99.130:58043
I20240426 09:49:03.886335 4061 raft_consensus.cc:482] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:03.886453 4061 raft_consensus.cc:3037] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:03.887676 4061 raft_consensus.cc:504] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:49:03.888670 4061 leader_election.cc:288] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [CANDIDATE]: Term 2 election: Requested vote from peers 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:49:03.889097 2529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7de16751060b4b91b679ba22803ae980" candidate_uuid: "3b591a3ffd514de7959be53199789ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ad0612beac445e896fcfc0562baa296"
I20240426 09:49:03.889269 2529 raft_consensus.cc:3032] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:03.889333 2529 raft_consensus.cc:729] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:49:03.889498 2529 consensus_queue.cc:260] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:49:03.889730 2529 raft_consensus.cc:3037] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:03.890652 2529 raft_consensus.cc:2454] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b591a3ffd514de7959be53199789ce8 in term 2.
I20240426 09:49:03.891103 3996 leader_election.cc:302] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296, 3b591a3ffd514de7959be53199789ce8; no voters:
I20240426 09:49:03.891489 4673 raft_consensus.cc:2781] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:03.891691 4673 raft_consensus.cc:686] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 2 LEADER]: Becoming Leader. State: Replica: 3b591a3ffd514de7959be53199789ce8, State: Running, Role: LEADER
I20240426 09:49:03.892287 4673 consensus_queue.cc:237] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:49:03.894644 2250 catalog_manager.cc:5484] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 reported cstate change: term changed from 1 to 2, leader changed from 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) to 3b591a3ffd514de7959be53199789ce8 (127.0.99.135). New cstate: current_term: 2 leader_uuid: "3b591a3ffd514de7959be53199789ce8" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:49:03.913894 2509 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:03.914569 4035 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:03.918696 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.926964 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.929489 3779 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:03.929783 2378 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:03.943157 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:03.967255 2398 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "54dd16182f294fe9af3ff82cd57b7e5a"
dest_uuid: "506a29f8a6094b909536aa04d980679d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60938
I20240426 09:49:03.967433 2398 raft_consensus.cc:595] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:03.978605 4490 raft_consensus.cc:979] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: : Instructing follower 400c68b80d144bd1b094c5cae03a111a to start an election
I20240426 09:49:03.978713 4490 raft_consensus.cc:1067] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Signalling peer 400c68b80d144bd1b094c5cae03a111a to start an election
I20240426 09:49:03.979383 3799 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "87ca4c907ced49ecb4b8acc48efe734d"
dest_uuid: "400c68b80d144bd1b094c5cae03a111a"
from {username='slave'} at 127.0.99.129:60725
I20240426 09:49:03.979579 3799 raft_consensus.cc:482] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:03.979693 3799 raft_consensus.cc:3037] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:03.980530 3799 raft_consensus.cc:504] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:49:03.981411 3799 leader_election.cc:288] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [CANDIDATE]: Term 2 election: Requested vote from peers 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:49:03.981945 2398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87ca4c907ced49ecb4b8acc48efe734d" candidate_uuid: "400c68b80d144bd1b094c5cae03a111a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "506a29f8a6094b909536aa04d980679d"
I20240426 09:49:03.982121 2398 raft_consensus.cc:3032] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:03.982178 2398 raft_consensus.cc:729] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:49:03.982321 2398 consensus_queue.cc:260] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:49:03.982548 2398 raft_consensus.cc:3037] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:03.983407 2398 raft_consensus.cc:2454] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 400c68b80d144bd1b094c5cae03a111a in term 2.
I20240426 09:49:03.983785 3732 leader_election.cc:302] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 400c68b80d144bd1b094c5cae03a111a, 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:49:03.984081 4705 raft_consensus.cc:2781] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:03.984234 4705 raft_consensus.cc:686] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 2 LEADER]: Becoming Leader. State: Replica: 400c68b80d144bd1b094c5cae03a111a, State: Running, Role: LEADER
I20240426 09:49:03.984529 4705 consensus_queue.cc:237] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:49:03.986881 2247 catalog_manager.cc:5484] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a reported cstate change: term changed from 1 to 2, leader changed from 506a29f8a6094b909536aa04d980679d (127.0.99.129) to 400c68b80d144bd1b094c5cae03a111a (127.0.99.133). New cstate: current_term: 2 leader_uuid: "400c68b80d144bd1b094c5cae03a111a" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:49:04.022689 2509 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:04.024240 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.026118 2378 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:04.031983 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:04.036425 4035 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.049341 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:04.050705 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.127506 4035 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.129508 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:04.130573 2509 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:04.130712 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.144484 2378 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:04.153564 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:04.161381 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.189702 2398 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c4810d8df46c4404875a4a53a50ecb60"
dest_uuid: "506a29f8a6094b909536aa04d980679d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60982
I20240426 09:49:04.189836 2398 raft_consensus.cc:595] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:04.239192 3910 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:04.240077 2509 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:04.241078 2378 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:04.242940 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.244637 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:04.259402 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.269603 4035 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.275405 4490 raft_consensus.cc:979] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: : Instructing follower 010c57b7730449c3b8c982a057e6268b to start an election
I20240426 09:49:04.275501 4490 raft_consensus.cc:1067] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Signalling peer 010c57b7730449c3b8c982a057e6268b to start an election
I20240426 09:49:04.275987 3930 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c4810d8df46c4404875a4a53a50ecb60"
dest_uuid: "010c57b7730449c3b8c982a057e6268b"
from {username='slave'} at 127.0.99.129:33899
I20240426 09:49:04.276173 3930 raft_consensus.cc:482] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:04.276268 3930 raft_consensus.cc:3037] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:04.277487 3930 raft_consensus.cc:504] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:49:04.278465 3930 leader_election.cc:288] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [CANDIDATE]: Term 2 election: Requested vote from peers 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:49:04.279012 2398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4810d8df46c4404875a4a53a50ecb60" candidate_uuid: "010c57b7730449c3b8c982a057e6268b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "506a29f8a6094b909536aa04d980679d"
I20240426 09:49:04.279134 2398 raft_consensus.cc:3032] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:04.279175 2398 raft_consensus.cc:729] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:49:04.279315 2398 consensus_queue.cc:260] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:49:04.279441 2398 raft_consensus.cc:3037] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:04.280368 2398 raft_consensus.cc:2454] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010c57b7730449c3b8c982a057e6268b in term 2.
I20240426 09:49:04.280715 3863 leader_election.cc:302] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 010c57b7730449c3b8c982a057e6268b, 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:49:04.281038 4807 raft_consensus.cc:2781] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:04.281200 4807 raft_consensus.cc:686] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 2 LEADER]: Becoming Leader. State: Replica: 010c57b7730449c3b8c982a057e6268b, State: Running, Role: LEADER
I20240426 09:49:04.281491 4807 consensus_queue.cc:237] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:49:04.283907 2255 catalog_manager.cc:5484] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b reported cstate change: term changed from 1 to 2, leader changed from 506a29f8a6094b909536aa04d980679d (127.0.99.129) to 010c57b7730449c3b8c982a057e6268b (127.0.99.134). New cstate: current_term: 2 leader_uuid: "010c57b7730449c3b8c982a057e6268b" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:49:04.301309 4490 raft_consensus.cc:979] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: : Instructing follower 010c57b7730449c3b8c982a057e6268b to start an election
I20240426 09:49:04.301409 4490 raft_consensus.cc:1067] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Signalling peer 010c57b7730449c3b8c982a057e6268b to start an election
I20240426 09:49:04.301834 3930 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "54dd16182f294fe9af3ff82cd57b7e5a"
dest_uuid: "010c57b7730449c3b8c982a057e6268b"
from {username='slave'} at 127.0.99.129:33899
I20240426 09:49:04.301983 3930 raft_consensus.cc:482] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:04.302039 3930 raft_consensus.cc:3037] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:04.302901 3930 raft_consensus.cc:504] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:49:04.303150 3930 leader_election.cc:288] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [CANDIDATE]: Term 2 election: Requested vote from peers 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:49:04.303526 2398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54dd16182f294fe9af3ff82cd57b7e5a" candidate_uuid: "010c57b7730449c3b8c982a057e6268b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "506a29f8a6094b909536aa04d980679d"
I20240426 09:49:04.303647 2398 raft_consensus.cc:3032] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:04.303695 2398 raft_consensus.cc:729] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:49:04.303828 2398 consensus_queue.cc:260] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:49:04.303946 2398 raft_consensus.cc:3037] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:04.304674 2398 raft_consensus.cc:2454] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 010c57b7730449c3b8c982a057e6268b in term 2.
I20240426 09:49:04.304940 3863 leader_election.cc:302] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 010c57b7730449c3b8c982a057e6268b, 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:49:04.305107 4807 raft_consensus.cc:2781] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:04.305198 4807 raft_consensus.cc:686] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 2 LEADER]: Becoming Leader. State: Replica: 010c57b7730449c3b8c982a057e6268b, State: Running, Role: LEADER
I20240426 09:49:04.305322 4807 consensus_queue.cc:237] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:49:04.306825 2255 catalog_manager.cc:5484] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b reported cstate change: term changed from 1 to 2, leader changed from 506a29f8a6094b909536aa04d980679d (127.0.99.129) to 010c57b7730449c3b8c982a057e6268b (127.0.99.134). New cstate: current_term: 2 leader_uuid: "010c57b7730449c3b8c982a057e6268b" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:49:04.339680 2529 raft_consensus.cc:1261] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Refusing update from remote peer 3b591a3ffd514de7959be53199789ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:04.340292 4673 consensus_queue.cc:1035] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:04.356290 4035 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.359807 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.364989 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:04.369709 2509 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:04.375847 2378 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:04.377025 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.387230 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:04.410485 2529 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "469ea537fdbc46d8982f23b6a1a181ab"
dest_uuid: "0ad0612beac445e896fcfc0562baa296"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:57386
I20240426 09:49:04.410627 2529 raft_consensus.cc:595] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:04.461421 2509 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:04.463197 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:04.463848 4035 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.465852 2378 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:04.474470 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:04.477834 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.499644 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.515996 2398 raft_consensus.cc:1261] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Refusing update from remote peer 400c68b80d144bd1b094c5cae03a111a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:04.516628 4898 consensus_queue.cc:1035] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:04.577178 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:04.578214 4035 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.578749 2509 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:04.580893 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.583204 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.591545 3648 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:04.604861 2378 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:04.631779 2398 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0db2ff8ccef842a2ac866f85b0e05134"
dest_uuid: "506a29f8a6094b909536aa04d980679d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:32784
I20240426 09:49:04.631920 2398 raft_consensus.cc:595] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:04.632445 4667 raft_consensus.cc:979] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: : Instructing follower a18c2b34c33241afbd517065a5b797b6 to start an election
I20240426 09:49:04.632525 4667 raft_consensus.cc:1067] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Signalling peer a18c2b34c33241afbd517065a5b797b6 to start an election
I20240426 09:49:04.633019 3657 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "469ea537fdbc46d8982f23b6a1a181ab"
dest_uuid: "a18c2b34c33241afbd517065a5b797b6"
from {username='slave'} at 127.0.99.130:43927
I20240426 09:49:04.633208 3657 raft_consensus.cc:482] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:04.633335 3657 raft_consensus.cc:3037] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:04.634282 3657 raft_consensus.cc:504] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:49:04.635061 3657 leader_election.cc:288] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [CANDIDATE]: Term 2 election: Requested vote from peers 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:49:04.635367 2529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "469ea537fdbc46d8982f23b6a1a181ab" candidate_uuid: "a18c2b34c33241afbd517065a5b797b6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ad0612beac445e896fcfc0562baa296"
I20240426 09:49:04.635483 2529 raft_consensus.cc:3032] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:04.635533 2529 raft_consensus.cc:729] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:49:04.635679 2529 consensus_queue.cc:260] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:49:04.635826 2529 raft_consensus.cc:3037] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:04.636752 2529 raft_consensus.cc:2454] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a18c2b34c33241afbd517065a5b797b6 in term 2.
I20240426 09:49:04.637137 3603 leader_election.cc:302] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296, a18c2b34c33241afbd517065a5b797b6; no voters:
I20240426 09:49:04.637434 4952 raft_consensus.cc:2781] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:04.637661 4952 raft_consensus.cc:686] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 2 LEADER]: Becoming Leader. State: Replica: a18c2b34c33241afbd517065a5b797b6, State: Running, Role: LEADER
I20240426 09:49:04.637964 4952 consensus_queue.cc:237] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:49:04.640278 2256 catalog_manager.cc:5484] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 reported cstate change: term changed from 1 to 2, leader changed from 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) to a18c2b34c33241afbd517065a5b797b6 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "a18c2b34c33241afbd517065a5b797b6" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:49:04.684324 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:04.685485 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.688161 4035 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.688755 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.694890 2398 raft_consensus.cc:1261] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Refusing update from remote peer 010c57b7730449c3b8c982a057e6268b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:04.695434 2378 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:04.696144 4807 consensus_queue.cc:1035] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:04.702323 2509 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:04.704646 2398 raft_consensus.cc:1261] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Refusing update from remote peer 010c57b7730449c3b8c982a057e6268b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:04.705675 4807 consensus_queue.cc:1035] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:04.727798 3648 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.804366 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:04.805172 2509 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:04.807474 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.808472 2378 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:04.808820 4035 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.812916 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.813326 3648 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.850998 2529 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "388e1882272944a3893813caef6cd903"
dest_uuid: "0ad0612beac445e896fcfc0562baa296"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:57460
I20240426 09:49:04.851145 2529 raft_consensus.cc:595] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:04.902437 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.904171 4945 raft_consensus.cc:979] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: : Instructing follower 3b591a3ffd514de7959be53199789ce8 to start an election
I20240426 09:49:04.904263 4945 raft_consensus.cc:1067] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Signalling peer 3b591a3ffd514de7959be53199789ce8 to start an election
I20240426 09:49:04.905608 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:04.905771 4061 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "388e1882272944a3893813caef6cd903"
dest_uuid: "3b591a3ffd514de7959be53199789ce8"
from {username='slave'} at 127.0.99.130:58043
I20240426 09:49:04.905908 4061 raft_consensus.cc:482] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:04.905974 4061 raft_consensus.cc:3037] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:04.906239 2509 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:04.907025 4061 raft_consensus.cc:504] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:49:04.907335 4061 leader_election.cc:288] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [CANDIDATE]: Term 2 election: Requested vote from peers 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:49:04.907819 2529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "388e1882272944a3893813caef6cd903" candidate_uuid: "3b591a3ffd514de7959be53199789ce8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ad0612beac445e896fcfc0562baa296"
I20240426 09:49:04.907928 2529 raft_consensus.cc:3032] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:04.907963 2529 raft_consensus.cc:729] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:49:04.908084 2529 consensus_queue.cc:260] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:49:04.908196 2529 raft_consensus.cc:3037] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:04.909116 2529 raft_consensus.cc:2454] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b591a3ffd514de7959be53199789ce8 in term 2.
I20240426 09:49:04.909401 3996 leader_election.cc:302] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296, 3b591a3ffd514de7959be53199789ce8; no voters:
I20240426 09:49:04.909691 4674 raft_consensus.cc:2781] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:04.909837 4674 raft_consensus.cc:686] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 2 LEADER]: Becoming Leader. State: Replica: 3b591a3ffd514de7959be53199789ce8, State: Running, Role: LEADER
I20240426 09:49:04.909932 2378 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:04.909950 4674 consensus_queue.cc:237] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:49:04.911485 2250 catalog_manager.cc:5484] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 reported cstate change: term changed from 1 to 2, leader changed from 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) to 3b591a3ffd514de7959be53199789ce8 (127.0.99.135). New cstate: current_term: 2 leader_uuid: "3b591a3ffd514de7959be53199789ce8" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:49:04.915485 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:04.917397 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:04.939051 3648 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.012979 2509 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:05.015897 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.016208 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.018397 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.022500 3511 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.035952 2529 raft_consensus.cc:1261] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Refusing update from remote peer a18c2b34c33241afbd517065a5b797b6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:05.036562 4952 consensus_queue.cc:1035] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:05.042001 3648 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.041998 2378 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:05.062117 4490 raft_consensus.cc:979] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: : Instructing follower 55f9bcefe16745108099bdd94114d98f to start an election
I20240426 09:49:05.062237 4490 raft_consensus.cc:1067] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Signalling peer 55f9bcefe16745108099bdd94114d98f to start an election
I20240426 09:49:05.062692 3531 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0db2ff8ccef842a2ac866f85b0e05134"
dest_uuid: "55f9bcefe16745108099bdd94114d98f"
from {username='slave'} at 127.0.99.129:57379
I20240426 09:49:05.062922 3531 raft_consensus.cc:482] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:05.062994 3531 raft_consensus.cc:3037] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:05.063858 3531 raft_consensus.cc:504] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:49:05.064535 3531 leader_election.cc:288] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [CANDIDATE]: Term 2 election: Requested vote from peers 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:49:05.064914 2397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db2ff8ccef842a2ac866f85b0e05134" candidate_uuid: "55f9bcefe16745108099bdd94114d98f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "506a29f8a6094b909536aa04d980679d"
I20240426 09:49:05.065043 2397 raft_consensus.cc:3032] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:05.065085 2397 raft_consensus.cc:729] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:49:05.065219 2397 consensus_queue.cc:260] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:49:05.065366 2397 raft_consensus.cc:3037] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:05.066380 2397 raft_consensus.cc:2454] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55f9bcefe16745108099bdd94114d98f in term 2.
I20240426 09:49:05.066725 3463 leader_election.cc:302] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d, 55f9bcefe16745108099bdd94114d98f; no voters:
I20240426 09:49:05.067060 5100 raft_consensus.cc:2781] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:05.067224 5100 raft_consensus.cc:686] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 2 LEADER]: Becoming Leader. State: Replica: 55f9bcefe16745108099bdd94114d98f, State: Running, Role: LEADER
I20240426 09:49:05.067368 5100 consensus_queue.cc:237] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:49:05.069289 2255 catalog_manager.cc:5484] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f reported cstate change: term changed from 1 to 2, leader changed from 506a29f8a6094b909536aa04d980679d (127.0.99.129) to 55f9bcefe16745108099bdd94114d98f (127.0.99.131). New cstate: current_term: 2 leader_uuid: "55f9bcefe16745108099bdd94114d98f" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } } }
I20240426 09:49:05.074501 2529 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "43b0427d465142a1b7639dd269112b15"
dest_uuid: "0ad0612beac445e896fcfc0562baa296"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:57496
I20240426 09:49:05.074631 2529 raft_consensus.cc:595] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:05.125375 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.128217 2509 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:05.131139 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.132869 3648 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.134696 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.144551 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.151706 2378 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:05.234894 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.238039 2509 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:05.240558 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.243316 3648 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.243927 2378 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:05.244668 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.253010 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.284047 2397 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "faeca7ce592f4ffe806903cb915c75d4"
dest_uuid: "506a29f8a6094b909536aa04d980679d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:32868
I20240426 09:49:05.284191 2397 raft_consensus.cc:595] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:05.334913 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.341318 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.341354 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.341607 2509 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:05.341862 2378 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:05.350205 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.367286 3648 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.458281 4667 raft_consensus.cc:979] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: : Instructing follower a18c2b34c33241afbd517065a5b797b6 to start an election
I20240426 09:49:05.458402 4667 raft_consensus.cc:1067] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Signalling peer a18c2b34c33241afbd517065a5b797b6 to start an election
I20240426 09:49:05.458873 3657 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "43b0427d465142a1b7639dd269112b15"
dest_uuid: "a18c2b34c33241afbd517065a5b797b6"
from {username='slave'} at 127.0.99.130:43927
I20240426 09:49:05.459002 3657 raft_consensus.cc:482] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:05.459048 3657 raft_consensus.cc:3037] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:05.459831 3657 raft_consensus.cc:504] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:49:05.460095 3657 leader_election.cc:288] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [CANDIDATE]: Term 2 election: Requested vote from peers 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:49:05.460444 2529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43b0427d465142a1b7639dd269112b15" candidate_uuid: "a18c2b34c33241afbd517065a5b797b6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ad0612beac445e896fcfc0562baa296"
I20240426 09:49:05.460546 2529 raft_consensus.cc:3032] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:05.460590 2529 raft_consensus.cc:729] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:49:05.460729 2529 consensus_queue.cc:260] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:49:05.460845 2529 raft_consensus.cc:3037] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:05.461588 2529 raft_consensus.cc:2454] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a18c2b34c33241afbd517065a5b797b6 in term 2.
I20240426 09:49:05.461845 3603 leader_election.cc:302] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296, a18c2b34c33241afbd517065a5b797b6; no voters:
I20240426 09:49:05.462057 4953 raft_consensus.cc:2781] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:05.462136 4953 raft_consensus.cc:686] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 2 LEADER]: Becoming Leader. State: Replica: a18c2b34c33241afbd517065a5b797b6, State: Running, Role: LEADER
I20240426 09:49:05.462270 4953 consensus_queue.cc:237] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:49:05.463696 2252 catalog_manager.cc:5484] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 reported cstate change: term changed from 1 to 2, leader changed from 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) to a18c2b34c33241afbd517065a5b797b6 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "a18c2b34c33241afbd517065a5b797b6" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:49:05.507457 2529 raft_consensus.cc:1261] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Refusing update from remote peer 3b591a3ffd514de7959be53199789ce8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:05.507889 5058 consensus_queue.cc:1035] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:05.632193 2397 raft_consensus.cc:1261] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Refusing update from remote peer 55f9bcefe16745108099bdd94114d98f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:05.633016 5247 consensus_queue.cc:1035] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:05.644220 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.647272 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.649829 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.652443 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.654949 2378 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:05.659085 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:05.664736 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.724040 3799 consensus_queue.cc:237] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:49:05.724833 4898 raft_consensus.cc:2932] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:49:05.728116 2247 catalog_manager.cc:5484] T 87ca4c907ced49ecb4b8acc48efe734d P 400c68b80d144bd1b094c5cae03a111a reported cstate change: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "400c68b80d144bd1b094c5cae03a111a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:49:05.730890 2378 tablet_service.cc:1514] Processing DeleteTablet for tablet 87ca4c907ced49ecb4b8acc48efe734d with delete_type TABLET_DATA_TOMBSTONED (TS 506a29f8a6094b909536aa04d980679d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44620
I20240426 09:49:05.731407 5283 tablet_replica.cc:330] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: stopping tablet replica
I20240426 09:49:05.731724 5283 raft_consensus.cc:2229] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:05.731900 5283 raft_consensus.cc:2258] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:05.733012 5283 ts_tablet_manager.cc:1875] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:05.735817 5283 ts_tablet_manager.cc:1888] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:05.735927 5283 log.cc:1196] T 87ca4c907ced49ecb4b8acc48efe734d P 506a29f8a6094b909536aa04d980679d: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/87ca4c907ced49ecb4b8acc48efe734d
I20240426 09:49:05.736577 2240 catalog_manager.cc:4830] TS 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009): tablet 87ca4c907ced49ecb4b8acc48efe734d (table rebalance_test_table_0 [id=d357ae9b5bc84013aa8876bfd166273a]) successfully deleted
I20240426 09:49:05.786149 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.787938 3779 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:05.788122 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.789194 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:05.791680 2378 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:05.810966 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.820598 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.828316 4704 raft_consensus.cc:979] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: : Instructing follower 400c68b80d144bd1b094c5cae03a111a to start an election
I20240426 09:49:05.828442 4704 raft_consensus.cc:1067] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Signalling peer 400c68b80d144bd1b094c5cae03a111a to start an election
I20240426 09:49:05.828860 3799 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "faeca7ce592f4ffe806903cb915c75d4"
dest_uuid: "400c68b80d144bd1b094c5cae03a111a"
from {username='slave'} at 127.0.99.129:60725
I20240426 09:49:05.828987 3799 raft_consensus.cc:482] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:05.829054 3799 raft_consensus.cc:3037] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:05.829881 3799 raft_consensus.cc:504] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:49:05.830173 3799 leader_election.cc:288] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [CANDIDATE]: Term 2 election: Requested vote from peers 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:49:05.830523 2398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faeca7ce592f4ffe806903cb915c75d4" candidate_uuid: "400c68b80d144bd1b094c5cae03a111a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "506a29f8a6094b909536aa04d980679d"
I20240426 09:49:05.830649 2398 raft_consensus.cc:3032] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:05.830703 2398 raft_consensus.cc:729] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:49:05.830870 2398 consensus_queue.cc:260] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:49:05.831010 2398 raft_consensus.cc:3037] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:05.831969 2398 raft_consensus.cc:2454] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 400c68b80d144bd1b094c5cae03a111a in term 2.
I20240426 09:49:05.832232 3732 leader_election.cc:302] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 400c68b80d144bd1b094c5cae03a111a, 506a29f8a6094b909536aa04d980679d; no voters:
I20240426 09:49:05.832430 4898 raft_consensus.cc:2781] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:05.832506 4898 raft_consensus.cc:686] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 2 LEADER]: Becoming Leader. State: Replica: 400c68b80d144bd1b094c5cae03a111a, State: Running, Role: LEADER
I20240426 09:49:05.832633 4898 consensus_queue.cc:237] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:49:05.834156 2252 catalog_manager.cc:5484] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a reported cstate change: term changed from 1 to 2, leader changed from 506a29f8a6094b909536aa04d980679d (127.0.99.129) to 400c68b80d144bd1b094c5cae03a111a (127.0.99.133). New cstate: current_term: 2 leader_uuid: "400c68b80d144bd1b094c5cae03a111a" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:49:05.855572 4061 consensus_queue.cc:237] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:49:05.856364 5217 raft_consensus.cc:2932] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:49:05.858501 2252 catalog_manager.cc:5484] T 7de16751060b4b91b679ba22803ae980 P 3b591a3ffd514de7959be53199789ce8 reported cstate change: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "3b591a3ffd514de7959be53199789ce8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:49:05.861106 2509 tablet_service.cc:1514] Processing DeleteTablet for tablet 7de16751060b4b91b679ba22803ae980 with delete_type TABLET_DATA_TOMBSTONED (TS 0ad0612beac445e896fcfc0562baa296 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35040
I20240426 09:49:05.861608 5339 tablet_replica.cc:330] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: stopping tablet replica
I20240426 09:49:05.861940 5339 raft_consensus.cc:2229] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:05.862123 5339 raft_consensus.cc:2258] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:05.863176 5339 ts_tablet_manager.cc:1875] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:05.865442 5339 ts_tablet_manager.cc:1888] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:05.865517 5339 log.cc:1196] T 7de16751060b4b91b679ba22803ae980 P 0ad0612beac445e896fcfc0562baa296: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/7de16751060b4b91b679ba22803ae980
I20240426 09:49:05.866052 2242 catalog_manager.cc:4830] TS 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121): tablet 7de16751060b4b91b679ba22803ae980 (table rebalance_test_table_3 [id=42be3da92ccd4b128a37f478a31c77bc]) successfully deleted
I20240426 09:49:05.914852 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:05.920792 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.921690 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:05.923359 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.923617 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.945902 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.948757 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:05.985831 3930 consensus_queue.cc:237] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:49:05.986593 4808 raft_consensus.cc:2932] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:49:05.988834 2250 catalog_manager.cc:5484] T 54dd16182f294fe9af3ff82cd57b7e5a P 010c57b7730449c3b8c982a057e6268b reported cstate change: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "010c57b7730449c3b8c982a057e6268b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:49:05.991340 2378 tablet_service.cc:1514] Processing DeleteTablet for tablet 54dd16182f294fe9af3ff82cd57b7e5a with delete_type TABLET_DATA_TOMBSTONED (TS 506a29f8a6094b909536aa04d980679d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44620
I20240426 09:49:05.991572 5283 tablet_replica.cc:330] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: stopping tablet replica
I20240426 09:49:05.991710 5283 raft_consensus.cc:2229] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:05.991811 5283 raft_consensus.cc:2258] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:05.992281 5283 ts_tablet_manager.cc:1875] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:05.993885 5283 ts_tablet_manager.cc:1888] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:05.993947 5283 log.cc:1196] T 54dd16182f294fe9af3ff82cd57b7e5a P 506a29f8a6094b909536aa04d980679d: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/54dd16182f294fe9af3ff82cd57b7e5a
I20240426 09:49:05.994379 2240 catalog_manager.cc:4830] TS 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009): tablet 54dd16182f294fe9af3ff82cd57b7e5a (table rebalance_test_table_1 [id=4f3ba36f2bfb4b28991e7efdbfe5e664]) successfully deleted
I20240426 09:49:06.046254 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.047070 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.047628 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.049911 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.051537 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.063473 2529 raft_consensus.cc:1261] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Refusing update from remote peer a18c2b34c33241afbd517065a5b797b6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:06.063933 5426 consensus_queue.cc:1035] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:06.070420 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.071875 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.114086 3930 consensus_queue.cc:237] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } }
I20240426 09:49:06.114838 4808 raft_consensus.cc:2932] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:49:06.117380 2247 catalog_manager.cc:5484] T c4810d8df46c4404875a4a53a50ecb60 P 010c57b7730449c3b8c982a057e6268b reported cstate change: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "010c57b7730449c3b8c982a057e6268b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "010c57b7730449c3b8c982a057e6268b" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 41855 } } }
I20240426 09:49:06.120117 2378 tablet_service.cc:1514] Processing DeleteTablet for tablet c4810d8df46c4404875a4a53a50ecb60 with delete_type TABLET_DATA_TOMBSTONED (TS 506a29f8a6094b909536aa04d980679d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44620
I20240426 09:49:06.120378 5283 tablet_replica.cc:330] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: stopping tablet replica
I20240426 09:49:06.120513 5283 raft_consensus.cc:2229] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:06.120601 5283 raft_consensus.cc:2258] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:06.121049 5283 ts_tablet_manager.cc:1875] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:06.123147 5283 ts_tablet_manager.cc:1888] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:06.123203 5283 log.cc:1196] T c4810d8df46c4404875a4a53a50ecb60 P 506a29f8a6094b909536aa04d980679d: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/c4810d8df46c4404875a4a53a50ecb60
I20240426 09:49:06.123696 2240 catalog_manager.cc:4830] TS 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009): tablet c4810d8df46c4404875a4a53a50ecb60 (table rebalance_test_table_4 [id=994cca768a73422ab7316a1feafaebfb]) successfully deleted
I20240426 09:49:06.179826 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.181150 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.185730 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.198830 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.199187 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.199383 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.209322 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.249081 3657 consensus_queue.cc:237] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:49:06.249868 5429 raft_consensus.cc:2932] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:49:06.252264 2250 catalog_manager.cc:5484] T 469ea537fdbc46d8982f23b6a1a181ab P a18c2b34c33241afbd517065a5b797b6 reported cstate change: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "a18c2b34c33241afbd517065a5b797b6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:49:06.254779 2509 tablet_service.cc:1514] Processing DeleteTablet for tablet 469ea537fdbc46d8982f23b6a1a181ab with delete_type TABLET_DATA_TOMBSTONED (TS 0ad0612beac445e896fcfc0562baa296 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35040
I20240426 09:49:06.255005 5339 tablet_replica.cc:330] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: stopping tablet replica
I20240426 09:49:06.255136 5339 raft_consensus.cc:2229] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:06.255244 5339 raft_consensus.cc:2258] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:06.255754 5339 ts_tablet_manager.cc:1875] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:06.257409 5339 ts_tablet_manager.cc:1888] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:06.257474 5339 log.cc:1196] T 469ea537fdbc46d8982f23b6a1a181ab P 0ad0612beac445e896fcfc0562baa296: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/469ea537fdbc46d8982f23b6a1a181ab
I20240426 09:49:06.257938 2242 catalog_manager.cc:4830] TS 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121): tablet 469ea537fdbc46d8982f23b6a1a181ab (table rebalance_test_table_3 [id=42be3da92ccd4b128a37f478a31c77bc]) successfully deleted
I20240426 09:49:06.311733 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.316854 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.323884 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.324323 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.326081 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.326117 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.331832 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.385782 3531 consensus_queue.cc:237] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } }
I20240426 09:49:06.386411 5247 raft_consensus.cc:2932] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } } }
I20240426 09:49:06.388717 2255 catalog_manager.cc:5484] T 0db2ff8ccef842a2ac866f85b0e05134 P 55f9bcefe16745108099bdd94114d98f reported cstate change: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "55f9bcefe16745108099bdd94114d98f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "55f9bcefe16745108099bdd94114d98f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 44151 } } }
I20240426 09:49:06.391227 2378 tablet_service.cc:1514] Processing DeleteTablet for tablet 0db2ff8ccef842a2ac866f85b0e05134 with delete_type TABLET_DATA_TOMBSTONED (TS 506a29f8a6094b909536aa04d980679d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44620
I20240426 09:49:06.391445 5283 tablet_replica.cc:330] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: stopping tablet replica
I20240426 09:49:06.391582 5283 raft_consensus.cc:2229] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:06.391695 5283 raft_consensus.cc:2258] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:06.392388 5283 ts_tablet_manager.cc:1875] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:06.394449 5283 ts_tablet_manager.cc:1888] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:06.394529 5283 log.cc:1196] T 0db2ff8ccef842a2ac866f85b0e05134 P 506a29f8a6094b909536aa04d980679d: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/0db2ff8ccef842a2ac866f85b0e05134
I20240426 09:49:06.395310 2240 catalog_manager.cc:4830] TS 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009): tablet 0db2ff8ccef842a2ac866f85b0e05134 (table rebalance_test_table_4 [id=994cca768a73422ab7316a1feafaebfb]) successfully deleted
I20240426 09:49:06.400291 2398 raft_consensus.cc:1261] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Refusing update from remote peer 400c68b80d144bd1b094c5cae03a111a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:06.400710 5568 consensus_queue.cc:1035] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:06.444921 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.447326 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.451659 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.452862 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.455780 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.459841 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.485606 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.518273 4061 consensus_queue.cc:237] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } }
I20240426 09:49:06.519068 5217 raft_consensus.cc:2932] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:49:06.521669 2249 catalog_manager.cc:5484] T 388e1882272944a3893813caef6cd903 P 3b591a3ffd514de7959be53199789ce8 reported cstate change: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "3b591a3ffd514de7959be53199789ce8" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3b591a3ffd514de7959be53199789ce8" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 33487 } } }
I20240426 09:49:06.524123 2509 tablet_service.cc:1514] Processing DeleteTablet for tablet 388e1882272944a3893813caef6cd903 with delete_type TABLET_DATA_TOMBSTONED (TS 0ad0612beac445e896fcfc0562baa296 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35040
I20240426 09:49:06.524364 5339 tablet_replica.cc:330] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: stopping tablet replica
I20240426 09:49:06.524482 5339 raft_consensus.cc:2229] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:06.524570 5339 raft_consensus.cc:2258] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:06.525022 5339 ts_tablet_manager.cc:1875] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:06.526837 5339 ts_tablet_manager.cc:1888] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:06.526901 5339 log.cc:1196] T 388e1882272944a3893813caef6cd903 P 0ad0612beac445e896fcfc0562baa296: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/388e1882272944a3893813caef6cd903
I20240426 09:49:06.527355 2242 catalog_manager.cc:4830] TS 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121): tablet 388e1882272944a3893813caef6cd903 (table rebalance_test_table_1 [id=4f3ba36f2bfb4b28991e7efdbfe5e664]) successfully deleted
I20240426 09:49:06.585603 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.589859 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.591096 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.591557 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.596887 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.600553 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.618295 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.651895 3657 consensus_queue.cc:237] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } }
I20240426 09:49:06.652490 5429 raft_consensus.cc:2932] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:49:06.654455 2250 catalog_manager.cc:5484] T 43b0427d465142a1b7639dd269112b15 P a18c2b34c33241afbd517065a5b797b6 reported cstate change: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "a18c2b34c33241afbd517065a5b797b6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a18c2b34c33241afbd517065a5b797b6" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 36485 } } }
I20240426 09:49:06.656865 2509 tablet_service.cc:1514] Processing DeleteTablet for tablet 43b0427d465142a1b7639dd269112b15 with delete_type TABLET_DATA_TOMBSTONED (TS 0ad0612beac445e896fcfc0562baa296 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35040
I20240426 09:49:06.657078 5339 tablet_replica.cc:330] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: stopping tablet replica
I20240426 09:49:06.657198 5339 raft_consensus.cc:2229] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:06.657301 5339 raft_consensus.cc:2258] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:06.657968 5339 ts_tablet_manager.cc:1875] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:06.660019 5339 ts_tablet_manager.cc:1888] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:06.660104 5339 log.cc:1196] T 43b0427d465142a1b7639dd269112b15 P 0ad0612beac445e896fcfc0562baa296: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/43b0427d465142a1b7639dd269112b15
I20240426 09:49:06.660617 2242 catalog_manager.cc:4830] TS 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121): tablet 43b0427d465142a1b7639dd269112b15 (table rebalance_test_table_2 [id=b49d74ebc4dc4e349407e21ef058b318]) successfully deleted
I20240426 09:49:06.714565 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.716041 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.727730 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.731310 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:06.735781 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.737917 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.744599 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:06.784510 3799 consensus_queue.cc:237] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } }
I20240426 09:49:06.785115 5571 raft_consensus.cc:2932] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:49:06.787189 2256 catalog_manager.cc:5484] T faeca7ce592f4ffe806903cb915c75d4 P 400c68b80d144bd1b094c5cae03a111a reported cstate change: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "400c68b80d144bd1b094c5cae03a111a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "400c68b80d144bd1b094c5cae03a111a" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 45093 } } }
I20240426 09:49:06.789597 2378 tablet_service.cc:1514] Processing DeleteTablet for tablet faeca7ce592f4ffe806903cb915c75d4 with delete_type TABLET_DATA_TOMBSTONED (TS 506a29f8a6094b909536aa04d980679d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44620
I20240426 09:49:06.789815 5283 tablet_replica.cc:330] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: stopping tablet replica
I20240426 09:49:06.789950 5283 raft_consensus.cc:2229] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:06.790048 5283 raft_consensus.cc:2258] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:06.790674 5283 ts_tablet_manager.cc:1875] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:06.792412 5283 ts_tablet_manager.cc:1888] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:06.792477 5283 log.cc:1196] T faeca7ce592f4ffe806903cb915c75d4 P 506a29f8a6094b909536aa04d980679d: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/faeca7ce592f4ffe806903cb915c75d4
I20240426 09:49:06.792966 2240 catalog_manager.cc:4830] TS 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009): tablet faeca7ce592f4ffe806903cb915c75d4 (table rebalance_test_table_2 [id=b49d74ebc4dc4e349407e21ef058b318]) successfully deleted
I20240426 09:49:07.094028 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.095763 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.097883 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.107769 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.108071 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.111793 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.121161 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.198072 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.200242 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.201570 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.202224 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.221032 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.221573 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.224470 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.444161 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.444931 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.449667 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.449837 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.450372 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.451757 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.457899 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.577301 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.578159 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.580523 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.587754 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.598327 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.601580 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.607815 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.696825 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.698695 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.702361 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.707834 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:07.711371 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.719506 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:07.725872 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
7e29b764f7de462a82e49984b426f327 | 127.0.99.190:33795 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:35031 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
010c57b7730449c3b8c982a057e6268b | 127.0.99.134:41855 | HEALTHY | <none> | 2 | 0
0ad0612beac445e896fcfc0562baa296 | 127.0.99.130:38121 | HEALTHY | <none> | 3 | 0
3b591a3ffd514de7959be53199789ce8 | 127.0.99.135:33487 | HEALTHY | <none> | 2 | 0
400c68b80d144bd1b094c5cae03a111a | 127.0.99.133:45093 | HEALTHY | <none> | 2 | 0
506a29f8a6094b909536aa04d980679d | 127.0.99.129:34009 | HEALTHY | <none> | 3 | 0
55f9bcefe16745108099bdd94114d98f | 127.0.99.131:44151 | HEALTHY | <none> | 2 | 0
a18c2b34c33241afbd517065a5b797b6 | 127.0.99.132:36485 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.0.99.129 | experimental | 127.0.99.129:34009
local_ip_for_outbound_sockets | 127.0.99.130 | experimental | 127.0.99.130:38121
local_ip_for_outbound_sockets | 127.0.99.131 | experimental | 127.0.99.131:44151
local_ip_for_outbound_sockets | 127.0.99.132 | experimental | 127.0.99.132:36485
local_ip_for_outbound_sockets | 127.0.99.133 | experimental | 127.0.99.133:45093
local_ip_for_outbound_sockets | 127.0.99.134 | experimental | 127.0.99.134:41855
local_ip_for_outbound_sockets | 127.0.99.135 | experimental | 127.0.99.135:33487
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.0.99.129:34009
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.0.99.130:38121
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.0.99.131:44151
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.0.99.132:36485
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.0.99.133:45093
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.0.99.134:41855
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.0.99.135:33487
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:35031 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 3
Maximum | 3
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240426 09:49:07.751410 398 log_verifier.cc:126] Checking tablet 0db2ff8ccef842a2ac866f85b0e05134
I20240426 09:49:07.762559 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 0db2ff8ccef842a2ac866f85b0e05134
I20240426 09:49:07.762651 398 log_verifier.cc:126] Checking tablet 1900ef96bb49402c9c757918f7fe3a41
I20240426 09:49:07.770946 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1900ef96bb49402c9c757918f7fe3a41
I20240426 09:49:07.771006 398 log_verifier.cc:126] Checking tablet 1ef96c18a0c8468e8d7668e05d7c4eea
I20240426 09:49:07.780146 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1ef96c18a0c8468e8d7668e05d7c4eea
I20240426 09:49:07.780210 398 log_verifier.cc:126] Checking tablet 248ceb0acd6c445fb2644fcc2c4f5b55
I20240426 09:49:07.788908 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 248ceb0acd6c445fb2644fcc2c4f5b55
I20240426 09:49:07.788970 398 log_verifier.cc:126] Checking tablet 31de0dcd76d94411ad4a0e2968960f26
I20240426 09:49:07.796849 398 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 31de0dcd76d94411ad4a0e2968960f26
I20240426 09:49:07.796903 398 log_verifier.cc:126] Checking tablet 388e1882272944a3893813caef6cd903
I20240426 09:49:07.805548 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 388e1882272944a3893813caef6cd903
I20240426 09:49:07.805611 398 log_verifier.cc:126] Checking tablet 43b0427d465142a1b7639dd269112b15
I20240426 09:49:07.814563 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 43b0427d465142a1b7639dd269112b15
I20240426 09:49:07.814625 398 log_verifier.cc:126] Checking tablet 469ea537fdbc46d8982f23b6a1a181ab
I20240426 09:49:07.823086 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 469ea537fdbc46d8982f23b6a1a181ab
I20240426 09:49:07.823158 398 log_verifier.cc:126] Checking tablet 54dd16182f294fe9af3ff82cd57b7e5a
I20240426 09:49:07.831626 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 54dd16182f294fe9af3ff82cd57b7e5a
I20240426 09:49:07.831687 398 log_verifier.cc:126] Checking tablet 60af15bf6ca4441e9718d44380392ff0
I20240426 09:49:07.840302 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 60af15bf6ca4441e9718d44380392ff0
I20240426 09:49:07.840374 398 log_verifier.cc:126] Checking tablet 62547c20121048a09eda5cf3014bbff6
I20240426 09:49:07.849056 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 62547c20121048a09eda5cf3014bbff6
I20240426 09:49:07.849121 398 log_verifier.cc:126] Checking tablet 7de16751060b4b91b679ba22803ae980
I20240426 09:49:07.858045 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 7de16751060b4b91b679ba22803ae980
I20240426 09:49:07.858121 398 log_verifier.cc:126] Checking tablet 80ada1082b6842f59c830591b94bd0e5
I20240426 09:49:07.866744 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 80ada1082b6842f59c830591b94bd0e5
I20240426 09:49:07.866844 398 log_verifier.cc:126] Checking tablet 87ca4c907ced49ecb4b8acc48efe734d
I20240426 09:49:07.875981 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 87ca4c907ced49ecb4b8acc48efe734d
I20240426 09:49:07.876046 398 log_verifier.cc:126] Checking tablet c4810d8df46c4404875a4a53a50ecb60
I20240426 09:49:07.884872 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet c4810d8df46c4404875a4a53a50ecb60
I20240426 09:49:07.884927 398 log_verifier.cc:126] Checking tablet faeca7ce592f4ffe806903cb915c75d4
I20240426 09:49:07.893774 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet faeca7ce592f4ffe806903cb915c75d4
I20240426 09:49:07.895787 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:0
--local_ip_for_outbound_sockets=127.0.99.136
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--builtin_ntp_servers=127.0.99.148:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:49:08.004248 5938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:08.004472 5938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:08.004514 5938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:08.008248 5938 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:08.008327 5938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:08.008416 5938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240426 09:49:08.013443 5938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:33795
--never_fsync=true
--heap_profile_path=/tmp/kudu.5938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:08.014458 5938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:08.015746 5938 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:08.024798 5946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:08.025574 5943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:08.026456 5938 server_base.cc:1034] running on GCE node
W20240426 09:49:08.027091 5944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:08.029767 5938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:08.030546 5938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:08.033833 5938 hybrid_clock.cc:648] HybridClock initialized: now 1714124948033800 us; error 50 us; skew 500 ppm
I20240426 09:49:08.035753 5938 webserver.cc:460] Webserver started at http://127.0.99.136:36125/ using document root <none> and password file <none>
I20240426 09:49:08.036307 5938 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:08.036365 5938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:08.036541 5938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:08.038179 5938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "e3408c8db85a47999e2bf43be4e38616"
format_stamp: "Formatted at 2024-04-26 09:49:08 on dist-test-slave-1zqj"
I20240426 09:49:08.038648 5938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "e3408c8db85a47999e2bf43be4e38616"
format_stamp: "Formatted at 2024-04-26 09:49:08 on dist-test-slave-1zqj"
I20240426 09:49:08.042099 5938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240426 09:49:08.044407 5952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:08.045485 5938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:08.045589 5938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e3408c8db85a47999e2bf43be4e38616"
format_stamp: "Formatted at 2024-04-26 09:49:08 on dist-test-slave-1zqj"
I20240426 09:49:08.045680 5938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:08.066154 5938 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:08.066807 5938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:08.066972 5938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:08.067519 5938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:08.068473 5938 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:08.068522 5938 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:08.068578 5938 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:08.068610 5938 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:08.078954 5938 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:41367
I20240426 09:49:08.078975 6065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:41367 every 8 connection(s)
I20240426 09:49:08.079914 5938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240426 09:49:08.082089 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 5938
I20240426 09:49:08.082214 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240426 09:49:08.091166 6066 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33795
I20240426 09:49:08.091424 6066 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:08.092039 6066 heartbeater.cc:511] Master 127.0.99.190:33795 requested a full tablet report, sending...
I20240426 09:49:08.093164 2249 ts_manager.cc:194] Registered new tserver with Master: e3408c8db85a47999e2bf43be4e38616 (127.0.99.136:41367)
I20240426 09:49:08.093983 2249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:47533
I20240426 09:49:08.282289 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:08.282873 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:08.287937 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.289604 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.291025 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.294754 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.300774 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.315096 6000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:08.444512 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:08.445796 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.447250 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.451923 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.453052 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:08.466110 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.481284 6000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:08.487373 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:08.523320 2529 consensus_queue.cc:237] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
W20240426 09:49:08.528594 2464 consensus_peers.cc:485] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 -> Peer e3408c8db85a47999e2bf43be4e38616 (127.0.99.136:41367): Couldn't send request to peer e3408c8db85a47999e2bf43be4e38616. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 62547c20121048a09eda5cf3014bbff6. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:09.010442 6171 ts_tablet_manager.cc:902] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: Initiating tablet copy from peer 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:49:09.011266 6171 tablet_copy_client.cc:308] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:38121
I20240426 09:49:09.014930 2549 tablet_copy_service.cc:140] P 0ad0612beac445e896fcfc0562baa296: Received BeginTabletCopySession request for tablet 62547c20121048a09eda5cf3014bbff6 from peer e3408c8db85a47999e2bf43be4e38616 ({username='slave'} at 127.0.99.136:47377)
I20240426 09:49:09.015034 2549 tablet_copy_service.cc:161] P 0ad0612beac445e896fcfc0562baa296: Beginning new tablet copy session on tablet 62547c20121048a09eda5cf3014bbff6 from peer e3408c8db85a47999e2bf43be4e38616 at {username='slave'} at 127.0.99.136:47377: session id = e3408c8db85a47999e2bf43be4e38616-62547c20121048a09eda5cf3014bbff6
I20240426 09:49:09.015862 2549 tablet_copy_source_session.cc:205] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:09.018850 6171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62547c20121048a09eda5cf3014bbff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:09.023524 6171 tablet_copy_client.cc:783] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:09.023708 6171 tablet_copy_client.cc:647] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:09.025132 6171 tablet_copy_client.cc:523] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:09.026661 6171 tablet_bootstrap.cc:492] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: Bootstrap starting.
I20240426 09:49:09.031456 6171 log.cc:824] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:09.049373 6171 tablet_bootstrap.cc:492] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:49:09.049779 6171 tablet_bootstrap.cc:492] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: Bootstrap complete.
I20240426 09:49:09.050055 6171 ts_tablet_manager.cc:1367] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.000s
I20240426 09:49:09.052214 6171 raft_consensus.cc:348] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } }
I20240426 09:49:09.052846 6171 raft_consensus.cc:729] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3408c8db85a47999e2bf43be4e38616, State: Initialized, Role: FOLLOWER
I20240426 09:49:09.053157 6171 consensus_queue.cc:260] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:09.053797 6066 heartbeater.cc:503] Master 127.0.99.190:33795 was elected leader, sending a full tablet report...
I20240426 09:49:09.053932 6171 ts_tablet_manager.cc:1398] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20240426 09:49:09.054657 2549 tablet_copy_service.cc:342] P 0ad0612beac445e896fcfc0562baa296: Request end of tablet copy session e3408c8db85a47999e2bf43be4e38616-62547c20121048a09eda5cf3014bbff6 received from {username='slave'} at 127.0.99.136:47377
I20240426 09:49:09.054740 2549 tablet_copy_service.cc:433] P 0ad0612beac445e896fcfc0562baa296: ending tablet copy session e3408c8db85a47999e2bf43be4e38616-62547c20121048a09eda5cf3014bbff6 on tablet 62547c20121048a09eda5cf3014bbff6 with peer e3408c8db85a47999e2bf43be4e38616
I20240426 09:49:09.595743 6009 raft_consensus.cc:1203] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:09.596874 6181 raft_consensus.cc:2932] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e3408c8db85a47999e2bf43be4e38616 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:09.597504 6009 raft_consensus.cc:2932] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e3408c8db85a47999e2bf43be4e38616 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:09.598217 6181 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:57698 (request call id 0) took 1075 ms. Trace:
I20240426 09:49:09.598289 6181 rpcz_store.cc:276] 0426 09:49:08.523002 (+ 0us) service_pool.cc:168] Inserting onto call queue
0426 09:49:08.523055 (+ 53us) service_pool.cc:225] Handling call
0426 09:49:08.523778 (+ 723us) log.cc:842] Serialized 257 byte log entry
0426 09:49:09.598200 (+1074422us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":41,"raft.queue_time_us":101,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":100,"threads_started":2,"wal-append.queue_time_us":143,"wal-append.run_cpu_time_us":323,"wal-append.run_wall_time_us":1000398}
I20240426 09:49:09.599552 2247 catalog_manager.cc:5484] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 reported cstate change: config changed from index -1 to 4, VOTER e3408c8db85a47999e2bf43be4e38616 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "0ad0612beac445e896fcfc0562baa296" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:09.616520 2398 consensus_queue.cc:237] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
W20240426 09:49:09.620496 2333 consensus_peers.cc:485] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d -> Peer e3408c8db85a47999e2bf43be4e38616 (127.0.99.136:41367): Couldn't send request to peer e3408c8db85a47999e2bf43be4e38616. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 60af15bf6ca4441e9718d44380392ff0. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:10.167519 6210 ts_tablet_manager.cc:902] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: Initiating tablet copy from peer 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:49:10.168001 6210 tablet_copy_client.cc:308] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:34009
I20240426 09:49:10.171478 2418 tablet_copy_service.cc:140] P 506a29f8a6094b909536aa04d980679d: Received BeginTabletCopySession request for tablet 60af15bf6ca4441e9718d44380392ff0 from peer e3408c8db85a47999e2bf43be4e38616 ({username='slave'} at 127.0.99.136:42957)
I20240426 09:49:10.171577 2418 tablet_copy_service.cc:161] P 506a29f8a6094b909536aa04d980679d: Beginning new tablet copy session on tablet 60af15bf6ca4441e9718d44380392ff0 from peer e3408c8db85a47999e2bf43be4e38616 at {username='slave'} at 127.0.99.136:42957: session id = e3408c8db85a47999e2bf43be4e38616-60af15bf6ca4441e9718d44380392ff0
I20240426 09:49:10.172370 2418 tablet_copy_source_session.cc:205] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:10.173235 6210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60af15bf6ca4441e9718d44380392ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:10.175458 6210 tablet_copy_client.cc:783] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:10.175629 6210 tablet_copy_client.cc:647] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:10.176714 6210 tablet_copy_client.cc:523] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:10.177644 6210 tablet_bootstrap.cc:492] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: Bootstrap starting.
I20240426 09:49:10.194110 6210 tablet_bootstrap.cc:492] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 09:49:10.194339 6210 tablet_bootstrap.cc:492] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: Bootstrap complete.
I20240426 09:49:10.194471 6210 ts_tablet_manager.cc:1367] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20240426 09:49:10.194916 6210 raft_consensus.cc:348] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } }
I20240426 09:49:10.195087 6210 raft_consensus.cc:729] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3408c8db85a47999e2bf43be4e38616, State: Initialized, Role: FOLLOWER
I20240426 09:49:10.195159 6210 consensus_queue.cc:260] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:10.195437 6210 ts_tablet_manager.cc:1398] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240426 09:49:10.195986 2418 tablet_copy_service.cc:342] P 506a29f8a6094b909536aa04d980679d: Request end of tablet copy session e3408c8db85a47999e2bf43be4e38616-60af15bf6ca4441e9718d44380392ff0 received from {username='slave'} at 127.0.99.136:42957
I20240426 09:49:10.196062 2418 tablet_copy_service.cc:433] P 506a29f8a6094b909536aa04d980679d: ending tablet copy session e3408c8db85a47999e2bf43be4e38616-60af15bf6ca4441e9718d44380392ff0 on tablet 60af15bf6ca4441e9718d44380392ff0 with peer e3408c8db85a47999e2bf43be4e38616
I20240426 09:49:10.647060 6009 raft_consensus.cc:1203] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:10.648416 6009 raft_consensus.cc:2932] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e3408c8db85a47999e2bf43be4e38616 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:10.648664 6219 raft_consensus.cc:2932] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER e3408c8db85a47999e2bf43be4e38616 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:10.650101 6219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:33050 (request call id 0) took 1033 ms. Trace:
I20240426 09:49:10.650202 6219 rpcz_store.cc:276] 0426 09:49:09.616208 (+ 0us) service_pool.cc:168] Inserting onto call queue
0426 09:49:09.616259 (+ 51us) service_pool.cc:225] Handling call
0426 09:49:09.616929 (+ 670us) log.cc:842] Serialized 257 byte log entry
0426 09:49:10.650084 (+1033155us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":43,"raft.queue_time_us":98,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":90,"threads_started":2,"wal-append.queue_time_us":115,"wal-append.run_cpu_time_us":308,"wal-append.run_wall_time_us":1000361}
I20240426 09:49:10.650925 2256 catalog_manager.cc:5484] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 reported cstate change: config changed from index -1 to 4, VOTER e3408c8db85a47999e2bf43be4e38616 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "506a29f8a6094b909536aa04d980679d" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:10.709765 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.710835 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:10.712725 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.718539 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.720784 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.731666 6000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:10.738490 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:10.747393 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.769647 2398 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "60af15bf6ca4441e9718d44380392ff0"
dest_uuid: "506a29f8a6094b909536aa04d980679d"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:33080
I20240426 09:49:10.769796 2398 raft_consensus.cc:595] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:10.819384 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:10.820546 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.821486 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.827606 6000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:10.830753 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:10.831065 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.851012 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.851228 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.930471 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.936240 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.939046 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:10.940212 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.941557 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.942831 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:10.949342 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:10.965502 6000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:10.989533 2529 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "62547c20121048a09eda5cf3014bbff6"
dest_uuid: "0ad0612beac445e896fcfc0562baa296"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:57764
I20240426 09:49:10.989693 2529 raft_consensus.cc:595] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:11.040623 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.041260 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.042615 2509 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:11.047609 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.048581 2378 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:11.055321 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.073913 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.079391 6000 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:11.221880 6388 raft_consensus.cc:979] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: : Instructing follower e3408c8db85a47999e2bf43be4e38616 to start an election
I20240426 09:49:11.222010 6388 raft_consensus.cc:1067] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Signalling peer e3408c8db85a47999e2bf43be4e38616 to start an election
I20240426 09:49:11.222517 6009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "60af15bf6ca4441e9718d44380392ff0"
dest_uuid: "e3408c8db85a47999e2bf43be4e38616"
from {username='slave'} at 127.0.99.129:42217
I20240426 09:49:11.222699 6009 raft_consensus.cc:482] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:11.222848 6009 raft_consensus.cc:3037] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:11.223723 6009 raft_consensus.cc:504] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:11.224613 6009 leader_election.cc:288] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [CANDIDATE]: Term 2 election: Requested vote from peers 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009)
I20240426 09:49:11.224987 2398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60af15bf6ca4441e9718d44380392ff0" candidate_uuid: "e3408c8db85a47999e2bf43be4e38616" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "506a29f8a6094b909536aa04d980679d"
I20240426 09:49:11.225091 2398 raft_consensus.cc:3032] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:11.225124 2398 raft_consensus.cc:729] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 506a29f8a6094b909536aa04d980679d, State: Running, Role: LEADER
I20240426 09:49:11.225265 2398 consensus_queue.cc:260] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:11.225426 2398 raft_consensus.cc:3037] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:11.226166 2398 raft_consensus.cc:2454] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3408c8db85a47999e2bf43be4e38616 in term 2.
I20240426 09:49:11.226526 5953 leader_election.cc:302] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 506a29f8a6094b909536aa04d980679d, e3408c8db85a47999e2bf43be4e38616; no voters:
I20240426 09:49:11.226876 6389 raft_consensus.cc:2781] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:11.227072 6389 raft_consensus.cc:686] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 2 LEADER]: Becoming Leader. State: Replica: e3408c8db85a47999e2bf43be4e38616, State: Running, Role: LEADER
I20240426 09:49:11.227341 6389 consensus_queue.cc:237] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:11.229405 2256 catalog_manager.cc:5484] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 reported cstate change: term changed from 1 to 2, leader changed from 506a29f8a6094b909536aa04d980679d (127.0.99.129) to e3408c8db85a47999e2bf43be4e38616 (127.0.99.136). New cstate: current_term: 2 leader_uuid: "e3408c8db85a47999e2bf43be4e38616" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:11.298342 6181 raft_consensus.cc:979] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: : Instructing follower e3408c8db85a47999e2bf43be4e38616 to start an election
I20240426 09:49:11.298454 6181 raft_consensus.cc:1067] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Signalling peer e3408c8db85a47999e2bf43be4e38616 to start an election
I20240426 09:49:11.298918 6009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "62547c20121048a09eda5cf3014bbff6"
dest_uuid: "e3408c8db85a47999e2bf43be4e38616"
from {username='slave'} at 127.0.99.130:35491
I20240426 09:49:11.299069 6009 raft_consensus.cc:482] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:11.299124 6009 raft_consensus.cc:3037] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:11.299993 6009 raft_consensus.cc:504] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:11.300348 6009 leader_election.cc:288] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [CANDIDATE]: Term 2 election: Requested vote from peers 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121)
I20240426 09:49:11.300745 2529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62547c20121048a09eda5cf3014bbff6" candidate_uuid: "e3408c8db85a47999e2bf43be4e38616" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0ad0612beac445e896fcfc0562baa296"
I20240426 09:49:11.300871 2529 raft_consensus.cc:3032] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:11.300925 2529 raft_consensus.cc:729] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ad0612beac445e896fcfc0562baa296, State: Running, Role: LEADER
I20240426 09:49:11.301107 2529 consensus_queue.cc:260] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:11.301254 2529 raft_consensus.cc:3037] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:11.302024 2529 raft_consensus.cc:2454] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3408c8db85a47999e2bf43be4e38616 in term 2.
I20240426 09:49:11.302366 5955 leader_election.cc:302] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 0ad0612beac445e896fcfc0562baa296, e3408c8db85a47999e2bf43be4e38616; no voters:
I20240426 09:49:11.302592 6389 raft_consensus.cc:2781] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:11.302695 6389 raft_consensus.cc:686] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 2 LEADER]: Becoming Leader. State: Replica: e3408c8db85a47999e2bf43be4e38616, State: Running, Role: LEADER
I20240426 09:49:11.302855 6389 consensus_queue.cc:237] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:11.304437 2256 catalog_manager.cc:5484] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 reported cstate change: term changed from 1 to 2, leader changed from 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) to e3408c8db85a47999e2bf43be4e38616 (127.0.99.136). New cstate: current_term: 2 leader_uuid: "e3408c8db85a47999e2bf43be4e38616" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 } } peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:11.359846 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.365821 2378 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.370599 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.373746 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.374019 2509 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.379599 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.388631 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.391261 6000 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.471940 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.475939 2509 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.476827 2378 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.480226 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.481858 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.484092 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.488909 6000 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.504896 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.788494 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.793884 2509 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.796273 2398 raft_consensus.cc:1261] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Refusing update from remote peer e3408c8db85a47999e2bf43be4e38616: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:11.796813 6389 consensus_queue.cc:1035] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a29f8a6094b909536aa04d980679d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 34009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:11.797698 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.797799 2378 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.801496 2529 raft_consensus.cc:1261] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Refusing update from remote peer e3408c8db85a47999e2bf43be4e38616: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:11.801733 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.802642 6000 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.802874 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.802863 6390 consensus_queue.cc:1035] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad0612beac445e896fcfc0562baa296" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:11.804224 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.862800 6009 consensus_queue.cc:237] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:11.863405 6389 raft_consensus.cc:2932] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:11.865543 2252 catalog_manager.cc:5484] T 60af15bf6ca4441e9718d44380392ff0 P e3408c8db85a47999e2bf43be4e38616 reported cstate change: config changed from index 4 to 6, VOTER 506a29f8a6094b909536aa04d980679d (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "e3408c8db85a47999e2bf43be4e38616" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:11.868446 2378 tablet_service.cc:1514] Processing DeleteTablet for tablet 60af15bf6ca4441e9718d44380392ff0 with delete_type TABLET_DATA_TOMBSTONED (TS 506a29f8a6094b909536aa04d980679d not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44620
I20240426 09:49:11.868845 6543 tablet_replica.cc:330] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: stopping tablet replica
I20240426 09:49:11.868973 6543 raft_consensus.cc:2229] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:11.869091 6543 raft_consensus.cc:2258] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:11.870396 6543 ts_tablet_manager.cc:1875] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:11.872735 6543 ts_tablet_manager.cc:1888] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:11.872833 6543 log.cc:1196] T 60af15bf6ca4441e9718d44380392ff0 P 506a29f8a6094b909536aa04d980679d: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/60af15bf6ca4441e9718d44380392ff0
I20240426 09:49:11.873440 2240 catalog_manager.cc:4830] TS 506a29f8a6094b909536aa04d980679d (127.0.99.129:34009): tablet 60af15bf6ca4441e9718d44380392ff0 (table rebalance_test_table_3 [id=42be3da92ccd4b128a37f478a31c77bc]) successfully deleted
I20240426 09:49:11.922304 2378 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.930454 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.931962 2509 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.933357 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.933508 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.934228 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.939282 6000 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.951851 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:11.997756 6009 consensus_queue.cc:237] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } }
I20240426 09:49:11.998332 6389 raft_consensus.cc:2932] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:12.000496 2256 catalog_manager.cc:5484] T 62547c20121048a09eda5cf3014bbff6 P e3408c8db85a47999e2bf43be4e38616 reported cstate change: config changed from index 4 to 6, VOTER 0ad0612beac445e896fcfc0562baa296 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "e3408c8db85a47999e2bf43be4e38616" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e3408c8db85a47999e2bf43be4e38616" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 41367 } } }
I20240426 09:49:12.002820 2509 tablet_service.cc:1514] Processing DeleteTablet for tablet 62547c20121048a09eda5cf3014bbff6 with delete_type TABLET_DATA_TOMBSTONED (TS 0ad0612beac445e896fcfc0562baa296 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:35040
I20240426 09:49:12.003172 6592 tablet_replica.cc:330] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: stopping tablet replica
I20240426 09:49:12.003309 6592 raft_consensus.cc:2229] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:12.003439 6592 raft_consensus.cc:2258] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:12.004704 6592 ts_tablet_manager.cc:1875] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:12.006459 6592 ts_tablet_manager.cc:1888] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:12.006525 6592 log.cc:1196] T 62547c20121048a09eda5cf3014bbff6 P 0ad0612beac445e896fcfc0562baa296: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/62547c20121048a09eda5cf3014bbff6
I20240426 09:49:12.007050 2242 catalog_manager.cc:4830] TS 0ad0612beac445e896fcfc0562baa296 (127.0.99.130:38121): tablet 62547c20121048a09eda5cf3014bbff6 (table rebalance_test_table_4 [id=994cca768a73422ab7316a1feafaebfb]) successfully deleted
I20240426 09:49:12.265203 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.265617 2509 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.274708 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.275041 2378 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.275599 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.283288 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.285522 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.299713 6000 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.382735 2509 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.382738 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.383188 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.383955 2378 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.385584 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.386061 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.406517 6000 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.407440 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.499332 3910 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.501480 4035 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.501480 2509 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.502063 2378 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.510185 3779 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.510623 3648 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.527227 3511 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:12.530135 6000 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
7e29b764f7de462a82e49984b426f327 | 127.0.99.190:33795 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:35031 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
010c57b7730449c3b8c982a057e6268b | 127.0.99.134:41855 | HEALTHY | <none> | 2 | 0
0ad0612beac445e896fcfc0562baa296 | 127.0.99.130:38121 | HEALTHY | <none> | 2 | 0
3b591a3ffd514de7959be53199789ce8 | 127.0.99.135:33487 | HEALTHY | <none> | 2 | 0
400c68b80d144bd1b094c5cae03a111a | 127.0.99.133:45093 | HEALTHY | <none> | 2 | 0
506a29f8a6094b909536aa04d980679d | 127.0.99.129:34009 | HEALTHY | <none> | 2 | 0
55f9bcefe16745108099bdd94114d98f | 127.0.99.131:44151 | HEALTHY | <none> | 2 | 0
a18c2b34c33241afbd517065a5b797b6 | 127.0.99.132:36485 | HEALTHY | <none> | 2 | 0
e3408c8db85a47999e2bf43be4e38616 | 127.0.99.136:41367 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.0.99.129 | experimental | 127.0.99.129:34009
local_ip_for_outbound_sockets | 127.0.99.130 | experimental | 127.0.99.130:38121
local_ip_for_outbound_sockets | 127.0.99.131 | experimental | 127.0.99.131:44151
local_ip_for_outbound_sockets | 127.0.99.132 | experimental | 127.0.99.132:36485
local_ip_for_outbound_sockets | 127.0.99.133 | experimental | 127.0.99.133:45093
local_ip_for_outbound_sockets | 127.0.99.134 | experimental | 127.0.99.134:41855
local_ip_for_outbound_sockets | 127.0.99.135 | experimental | 127.0.99.135:33487
local_ip_for_outbound_sockets | 127.0.99.136 | experimental | 127.0.99.136:41367
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.0.99.129:34009
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.0.99.130:38121
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.0.99.131:44151
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.0.99.132:36485
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.0.99.133:45093
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.0.99.134:41855
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.0.99.135:33487
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.0.99.136:41367
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:35031 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 2
Maximum | 2
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240426 09:49:12.561964 398 log_verifier.cc:126] Checking tablet 0db2ff8ccef842a2ac866f85b0e05134
I20240426 09:49:12.568408 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 0db2ff8ccef842a2ac866f85b0e05134
I20240426 09:49:12.568473 398 log_verifier.cc:126] Checking tablet 1900ef96bb49402c9c757918f7fe3a41
I20240426 09:49:12.574306 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1900ef96bb49402c9c757918f7fe3a41
I20240426 09:49:12.574376 398 log_verifier.cc:126] Checking tablet 1ef96c18a0c8468e8d7668e05d7c4eea
I20240426 09:49:12.579901 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1ef96c18a0c8468e8d7668e05d7c4eea
I20240426 09:49:12.579957 398 log_verifier.cc:126] Checking tablet 248ceb0acd6c445fb2644fcc2c4f5b55
I20240426 09:49:12.585304 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 248ceb0acd6c445fb2644fcc2c4f5b55
I20240426 09:49:12.585354 398 log_verifier.cc:126] Checking tablet 31de0dcd76d94411ad4a0e2968960f26
I20240426 09:49:12.589955 398 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 31de0dcd76d94411ad4a0e2968960f26
I20240426 09:49:12.590003 398 log_verifier.cc:126] Checking tablet 388e1882272944a3893813caef6cd903
I20240426 09:49:12.595682 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 388e1882272944a3893813caef6cd903
I20240426 09:49:12.595741 398 log_verifier.cc:126] Checking tablet 43b0427d465142a1b7639dd269112b15
I20240426 09:49:12.601203 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 43b0427d465142a1b7639dd269112b15
I20240426 09:49:12.601261 398 log_verifier.cc:126] Checking tablet 469ea537fdbc46d8982f23b6a1a181ab
I20240426 09:49:12.606642 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 469ea537fdbc46d8982f23b6a1a181ab
I20240426 09:49:12.606686 398 log_verifier.cc:126] Checking tablet 54dd16182f294fe9af3ff82cd57b7e5a
I20240426 09:49:12.612077 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 54dd16182f294fe9af3ff82cd57b7e5a
I20240426 09:49:12.612123 398 log_verifier.cc:126] Checking tablet 60af15bf6ca4441e9718d44380392ff0
I20240426 09:49:12.620047 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 60af15bf6ca4441e9718d44380392ff0
I20240426 09:49:12.620105 398 log_verifier.cc:126] Checking tablet 62547c20121048a09eda5cf3014bbff6
I20240426 09:49:12.628391 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 62547c20121048a09eda5cf3014bbff6
I20240426 09:49:12.628449 398 log_verifier.cc:126] Checking tablet 7de16751060b4b91b679ba22803ae980
I20240426 09:49:12.634146 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 7de16751060b4b91b679ba22803ae980
I20240426 09:49:12.634207 398 log_verifier.cc:126] Checking tablet 80ada1082b6842f59c830591b94bd0e5
I20240426 09:49:12.639767 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 80ada1082b6842f59c830591b94bd0e5
I20240426 09:49:12.639823 398 log_verifier.cc:126] Checking tablet 87ca4c907ced49ecb4b8acc48efe734d
I20240426 09:49:12.645514 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 87ca4c907ced49ecb4b8acc48efe734d
I20240426 09:49:12.645571 398 log_verifier.cc:126] Checking tablet c4810d8df46c4404875a4a53a50ecb60
I20240426 09:49:12.651177 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet c4810d8df46c4404875a4a53a50ecb60
I20240426 09:49:12.651229 398 log_verifier.cc:126] Checking tablet faeca7ce592f4ffe806903cb915c75d4
I20240426 09:49:12.656864 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet faeca7ce592f4ffe806903cb915c75d4
I20240426 09:49:12.657346 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 2316
I20240426 09:49:12.664426 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 2447
I20240426 09:49:12.671173 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 3448
I20240426 09:49:12.677182 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 3580
I20240426 09:49:12.683058 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 3717
I20240426 09:49:12.688436 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 3848
I20240426 09:49:12.696664 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 3979
I20240426 09:49:12.702458 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 5938
I20240426 09:49:12.708076 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 2225
2024-04-26T09:49:12Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/0 (26780 ms)
[----------] 1 test from RebalanceParamTest (26780 ms total)
[----------] 1 test from DDLDuringRebalancingTest
[ RUN ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/0
I20240426 09:49:12.748703 398 test_util.cc:268] Using random seed: -790052201
I20240426 09:49:12.749598 398 ts_itest-base.cc:112] Starting cluster with:
I20240426 09:49:12.749651 398 ts_itest-base.cc:113] --------------
I20240426 09:49:12.749676 398 ts_itest-base.cc:114] 8 tablet servers
I20240426 09:49:12.749692 398 ts_itest-base.cc:115] 3 replicas per TS
I20240426 09:49:12.749715 398 ts_itest-base.cc:116] --------------
2024-04-26T09:49:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-04-26T09:49:12Z Disabled control of system clock
I20240426 09:49:12.758203 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:43685
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:43685
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240426 09:49:12.862951 6709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:12.863180 6709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:12.863219 6709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:12.866210 6709 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:12.866273 6709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240426 09:49:12.866297 6709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:12.866314 6709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240426 09:49:12.866331 6709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240426 09:49:12.869882 6709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:43685
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:43685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.6709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:12.870755 6709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:12.871853 6709 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:12.879822 6717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:12.880574 6714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:12.881438 6715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:12.882335 6709 server_base.cc:1034] running on GCE node
I20240426 09:49:12.882922 6709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:12.883873 6709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:12.885031 6709 hybrid_clock.cc:648] HybridClock initialized: now 1714124952885020 us; error 27 us; skew 500 ppm
I20240426 09:49:12.887013 6709 webserver.cc:460] Webserver started at http://127.0.99.190:37485/ using document root <none> and password file <none>
I20240426 09:49:12.887558 6709 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:12.887624 6709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:12.887823 6709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:12.889469 6709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "27345b30edb745aa91af53b9ce01d7da"
format_stamp: "Formatted at 2024-04-26 09:49:12 on dist-test-slave-1zqj"
I20240426 09:49:12.889948 6709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "27345b30edb745aa91af53b9ce01d7da"
format_stamp: "Formatted at 2024-04-26 09:49:12 on dist-test-slave-1zqj"
I20240426 09:49:12.893246 6709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20240426 09:49:12.895359 6723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:12.896353 6709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:12.896488 6709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "27345b30edb745aa91af53b9ce01d7da"
format_stamp: "Formatted at 2024-04-26 09:49:12 on dist-test-slave-1zqj"
I20240426 09:49:12.896577 6709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:12.913630 6709 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:12.914280 6709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:12.914459 6709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:12.920883 6709 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:43685
I20240426 09:49:12.920898 6775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:43685 every 8 connection(s)
I20240426 09:49:12.921877 6709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240426 09:49:12.924492 6776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:12.924553 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 6709
I20240426 09:49:12.924652 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240426 09:49:12.930794 6776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Bootstrap starting.
I20240426 09:49:12.933081 6776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:12.933912 6776 log.cc:824] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:12.935505 6776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: No bootstrap required, opened a new log
I20240426 09:49:12.937929 6776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27345b30edb745aa91af53b9ce01d7da" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 43685 } }
I20240426 09:49:12.938115 6776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:12.938182 6776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27345b30edb745aa91af53b9ce01d7da, State: Initialized, Role: FOLLOWER
I20240426 09:49:12.938603 6776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27345b30edb745aa91af53b9ce01d7da" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 43685 } }
I20240426 09:49:12.938738 6776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:49:12.938841 6776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:49:12.938968 6776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:12.939728 6776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27345b30edb745aa91af53b9ce01d7da" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 43685 } }
I20240426 09:49:12.940037 6776 leader_election.cc:302] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27345b30edb745aa91af53b9ce01d7da; no voters:
I20240426 09:49:12.940302 6776 leader_election.cc:288] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:49:12.940713 6781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:12.940907 6781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [term 1 LEADER]: Becoming Leader. State: Replica: 27345b30edb745aa91af53b9ce01d7da, State: Running, Role: LEADER
I20240426 09:49:12.941162 6781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27345b30edb745aa91af53b9ce01d7da" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 43685 } }
I20240426 09:49:12.941331 6776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [sys.catalog]: configured and running, proceeding with master startup.
I20240426 09:49:12.942548 6783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27345b30edb745aa91af53b9ce01d7da. Latest consensus state: current_term: 1 leader_uuid: "27345b30edb745aa91af53b9ce01d7da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27345b30edb745aa91af53b9ce01d7da" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 43685 } } }
I20240426 09:49:12.942689 6783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [sys.catalog]: This master's current role is: LEADER
I20240426 09:49:12.943013 6782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27345b30edb745aa91af53b9ce01d7da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27345b30edb745aa91af53b9ce01d7da" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 43685 } } }
I20240426 09:49:12.943145 6782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da [sys.catalog]: This master's current role is: LEADER
I20240426 09:49:12.943639 6790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 09:49:12.947067 6790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 09:49:12.952119 6790 catalog_manager.cc:1347] Generated new cluster ID: 987be04212db4208a64c55a2952f1efa
I20240426 09:49:12.952206 6790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 09:49:12.963819 6790 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 09:49:12.964720 6790 catalog_manager.cc:1504] Loading token signing keys...
I20240426 09:49:12.974514 6790 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Generated new TSK 0
I20240426 09:49:12.975255 6790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 09:49:12.982689 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:49:13.087095 6800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:13.087325 6800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:13.087400 6800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:13.090415 6800 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:13.090514 6800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:13.090623 6800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240426 09:49:13.094223 6800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.6800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:13.095166 6800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:13.096377 6800 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:13.105338 6806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.106262 6805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.107050 6808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:13.107955 6800 server_base.cc:1034] running on GCE node
I20240426 09:49:13.108343 6800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:13.108919 6800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:13.110069 6800 hybrid_clock.cc:648] HybridClock initialized: now 1714124953110059 us; error 29 us; skew 500 ppm
I20240426 09:49:13.112267 6800 webserver.cc:460] Webserver started at http://127.0.99.129:44659/ using document root <none> and password file <none>
I20240426 09:49:13.112838 6800 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:13.112895 6800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:13.113073 6800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:13.114722 6800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "c244e02f3cf847c0a5b69e1059f1071c"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.115237 6800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "c244e02f3cf847c0a5b69e1059f1071c"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.118494 6800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240426 09:49:13.120733 6814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.121707 6800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:13.121819 6800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "c244e02f3cf847c0a5b69e1059f1071c"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.121907 6800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:13.132014 6800 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:13.132612 6800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:13.132797 6800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:13.133313 6800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:13.134169 6800 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:13.134218 6800 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.134274 6800 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:13.134305 6800 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.142882 6800 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:41571
I20240426 09:49:13.142917 6927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:41571 every 8 connection(s)
I20240426 09:49:13.143828 6800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240426 09:49:13.148576 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 6800
I20240426 09:49:13.148684 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240426 09:49:13.150748 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:49:13.153064 6928 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:13.153290 6928 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:13.153749 6928 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:13.155441 6740 ts_manager.cc:194] Registered new tserver with Master: c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:13.157244 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:56587
W20240426 09:49:13.258684 6931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:13.258941 6931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:13.258991 6931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:13.262048 6931 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:13.262120 6931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:13.262189 6931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240426 09:49:13.265825 6931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.6931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:13.266793 6931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:13.267889 6931 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:13.276818 6939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.277521 6936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.278349 6937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:13.279345 6931 server_base.cc:1034] running on GCE node
I20240426 09:49:13.279742 6931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:13.280282 6931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:13.281421 6931 hybrid_clock.cc:648] HybridClock initialized: now 1714124953281413 us; error 26 us; skew 500 ppm
I20240426 09:49:13.283399 6931 webserver.cc:460] Webserver started at http://127.0.99.130:41615/ using document root <none> and password file <none>
I20240426 09:49:13.283932 6931 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:13.283984 6931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:13.284132 6931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:13.285750 6931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.286196 6931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.289448 6931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240426 09:49:13.291625 6945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.292589 6931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:13.292687 6931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.292758 6931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:13.306850 6931 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:13.307432 6931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:13.307571 6931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:13.308038 6931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:13.308892 6931 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:13.308939 6931 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.308979 6931 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:13.308992 6931 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.317065 6931 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:37135
I20240426 09:49:13.317085 7058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:37135 every 8 connection(s)
I20240426 09:49:13.317973 6931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240426 09:49:13.325997 7059 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:13.326220 7059 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:13.326628 7059 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:13.327140 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 6931
I20240426 09:49:13.327261 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240426 09:49:13.327613 6740 ts_manager.cc:194] Registered new tserver with Master: 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:13.328209 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:57329
I20240426 09:49:13.329327 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:49:13.433495 7062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:13.433692 7062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:13.433735 7062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:13.436704 7062 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:13.436766 7062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:13.436863 7062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240426 09:49:13.440311 7062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.7062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:13.441231 7062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:13.442389 7062 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:13.452039 7067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.452869 7068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.453635 7070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:13.454474 7062 server_base.cc:1034] running on GCE node
I20240426 09:49:13.454888 7062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:13.455430 7062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:13.456593 7062 hybrid_clock.cc:648] HybridClock initialized: now 1714124953456584 us; error 29 us; skew 500 ppm
I20240426 09:49:13.458475 7062 webserver.cc:460] Webserver started at http://127.0.99.131:45233/ using document root <none> and password file <none>
I20240426 09:49:13.459064 7062 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:13.459126 7062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:13.459297 7062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:13.460912 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "1aaf8befba984e199e96f53f46b50441"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.461407 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "1aaf8befba984e199e96f53f46b50441"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.464649 7062 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240426 09:49:13.466696 7076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.467661 7062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:13.467762 7062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "1aaf8befba984e199e96f53f46b50441"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.467849 7062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:13.491106 7062 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:13.491747 7062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:13.491912 7062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:13.492444 7062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:13.493356 7062 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:13.493407 7062 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.493469 7062 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:13.493500 7062 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.502313 7062 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:39021
I20240426 09:49:13.502348 7189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:39021 every 8 connection(s)
I20240426 09:49:13.503261 7062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240426 09:49:13.505044 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 7062
I20240426 09:49:13.505172 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240426 09:49:13.507490 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:49:13.512611 7190 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:13.512873 7190 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:13.513346 7190 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:13.514353 6739 ts_manager.cc:194] Registered new tserver with Master: 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:13.515004 6739 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:47611
W20240426 09:49:13.617355 7193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:13.617583 7193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:13.617658 7193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:13.620730 7193 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:13.620808 7193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:13.620904 7193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240426 09:49:13.624413 7193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.7193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:13.625353 7193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:13.626470 7193 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:13.635248 7201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.635960 7199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.636893 7198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:13.637766 7193 server_base.cc:1034] running on GCE node
I20240426 09:49:13.638212 7193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:13.638886 7193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:13.640065 7193 hybrid_clock.cc:648] HybridClock initialized: now 1714124953640055 us; error 29 us; skew 500 ppm
I20240426 09:49:13.642170 7193 webserver.cc:460] Webserver started at http://127.0.99.132:46219/ using document root <none> and password file <none>
I20240426 09:49:13.642789 7193 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:13.642859 7193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:13.643049 7193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:13.644728 7193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.645181 7193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.648442 7193 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240426 09:49:13.650532 7207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.651526 7193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:13.651629 7193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.651715 7193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:13.664618 7193 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:13.665222 7193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:13.665355 7193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:13.665834 7193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:13.666666 7193 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:13.666709 7193 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.666747 7193 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:13.666808 7193 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.674829 7193 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:38055
I20240426 09:49:13.674867 7320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:38055 every 8 connection(s)
I20240426 09:49:13.675757 7193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240426 09:49:13.683072 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 7193
I20240426 09:49:13.683194 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240426 09:49:13.684309 7321 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:13.684538 7321 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:13.684986 7321 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:13.685139 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:0
--local_ip_for_outbound_sockets=127.0.99.133
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:49:13.685935 6739 ts_manager.cc:194] Registered new tserver with Master: e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:13.686619 6739 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:38203
W20240426 09:49:13.792341 7324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:13.792580 7324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:13.792658 7324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:13.795727 7324 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:13.795806 7324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:13.795900 7324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240426 09:49:13.799597 7324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.7324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:13.800572 7324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:13.801730 7324 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:13.810753 7329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.811642 7332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:13.812512 7330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:13.813347 7324 server_base.cc:1034] running on GCE node
I20240426 09:49:13.813763 7324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:13.814349 7324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:13.815493 7324 hybrid_clock.cc:648] HybridClock initialized: now 1714124953815483 us; error 27 us; skew 500 ppm
I20240426 09:49:13.817517 7324 webserver.cc:460] Webserver started at http://127.0.99.133:40677/ using document root <none> and password file <none>
I20240426 09:49:13.818045 7324 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:13.818097 7324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:13.818256 7324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:13.820056 7324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "a692602e1a2e4794877e6fb92a7b9962"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.820585 7324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "a692602e1a2e4794877e6fb92a7b9962"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.824023 7324 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240426 09:49:13.826197 7338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.827255 7324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:13.827356 7324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "a692602e1a2e4794877e6fb92a7b9962"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:13.827451 7324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:13.851161 7324 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:13.851784 7324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:13.851923 7324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:13.852419 7324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:13.853323 7324 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:13.853369 7324 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.853408 7324 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:13.853428 7324 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:13.861785 7324 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:40545
I20240426 09:49:13.861806 7451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:40545 every 8 connection(s)
I20240426 09:49:13.862754 7324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240426 09:49:13.871062 7452 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:13.871294 7452 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:13.871471 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 7324
I20240426 09:49:13.871583 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240426 09:49:13.871845 7452 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:13.872866 6740 ts_manager.cc:194] Registered new tserver with Master: a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:13.873524 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:52489
I20240426 09:49:13.873832 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:0
--local_ip_for_outbound_sockets=127.0.99.134
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:49:13.988235 7455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:13.988495 7455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:13.988582 7455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:13.991674 7455 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:13.991765 7455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:13.991869 7455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240426 09:49:13.995553 7455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.7455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:13.996557 7455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:13.997866 7455 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:14.007002 7460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:14.007758 7463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:14.008678 7461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:14.009536 7455 server_base.cc:1034] running on GCE node
I20240426 09:49:14.009924 7455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:14.010475 7455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:14.011739 7455 hybrid_clock.cc:648] HybridClock initialized: now 1714124954011701 us; error 70 us; skew 500 ppm
I20240426 09:49:14.013705 7455 webserver.cc:460] Webserver started at http://127.0.99.134:37725/ using document root <none> and password file <none>
I20240426 09:49:14.014264 7455 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:14.014344 7455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:14.014513 7455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:14.016184 7455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "3b41d87e7bfb419f8aa7500100129b6c"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:14.016638 7455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "3b41d87e7bfb419f8aa7500100129b6c"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:14.019896 7455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240426 09:49:14.021890 7469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:14.022907 7455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240426 09:49:14.023008 7455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "3b41d87e7bfb419f8aa7500100129b6c"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:14.023092 7455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:14.036442 7455 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:14.037055 7455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:14.037215 7455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:14.037755 7455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:14.038666 7455 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:14.038722 7455 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:14.038800 7455 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:14.038831 7455 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:14.047093 7455 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:37203
I20240426 09:49:14.047111 7582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:37203 every 8 connection(s)
I20240426 09:49:14.048039 7455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240426 09:49:14.049479 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 7455
I20240426 09:49:14.049626 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240426 09:49:14.051833 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:0
--local_ip_for_outbound_sockets=127.0.99.135
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:49:14.057372 7583 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:14.057600 7583 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:14.058041 7583 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:14.059008 6740 ts_manager.cc:194] Registered new tserver with Master: 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:14.059546 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:41867
I20240426 09:49:14.159775 6928 heartbeater.cc:503] Master 127.0.99.190:43685 was elected leader, sending a full tablet report...
W20240426 09:49:14.161978 7586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:14.162232 7586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:14.162286 7586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:14.165495 7586 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:14.165572 7586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:14.165643 7586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240426 09:49:14.169461 7586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.7586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:14.170415 7586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:14.171629 7586 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:14.180490 7594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:14.181306 7591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:14.182121 7592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:14.183034 7586 server_base.cc:1034] running on GCE node
I20240426 09:49:14.183389 7586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:14.183993 7586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:14.185137 7586 hybrid_clock.cc:648] HybridClock initialized: now 1714124954185128 us; error 24 us; skew 500 ppm
I20240426 09:49:14.187104 7586 webserver.cc:460] Webserver started at http://127.0.99.135:40803/ using document root <none> and password file <none>
I20240426 09:49:14.187647 7586 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:14.187701 7586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:14.187877 7586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:14.189491 7586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:14.189961 7586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:14.193195 7586 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20240426 09:49:14.195292 7600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:14.196306 7586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:14.196408 7586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:14.196493 7586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:14.210644 7586 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:14.211264 7586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:14.211419 7586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:14.211937 7586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:14.212854 7586 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:14.212908 7586 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:14.212970 7586 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:14.212996 7586 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:14.220992 7586 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:36979
I20240426 09:49:14.221016 7713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:36979 every 8 connection(s)
I20240426 09:49:14.221894 7586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240426 09:49:14.228322 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 7586
I20240426 09:49:14.228451 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240426 09:49:14.230824 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:0
--local_ip_for_outbound_sockets=127.0.99.136
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:49:14.232157 7714 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:14.232394 7714 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:14.232889 7714 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:14.233899 6740 ts_manager.cc:194] Registered new tserver with Master: b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:14.234705 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:54407
I20240426 09:49:14.330075 7059 heartbeater.cc:503] Master 127.0.99.190:43685 was elected leader, sending a full tablet report...
W20240426 09:49:14.337370 7717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:14.337581 7717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:14.337622 7717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:14.340536 7717 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:14.340595 7717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:14.340658 7717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240426 09:49:14.344166 7717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.7717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:14.345008 7717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:14.346064 7717 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:14.354686 7722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:14.355485 7725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:14.356274 7723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:14.357149 7717 server_base.cc:1034] running on GCE node
I20240426 09:49:14.357556 7717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:14.358124 7717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:14.359274 7717 hybrid_clock.cc:648] HybridClock initialized: now 1714124954359265 us; error 32 us; skew 500 ppm
I20240426 09:49:14.361258 7717 webserver.cc:460] Webserver started at http://127.0.99.136:34323/ using document root <none> and password file <none>
I20240426 09:49:14.361819 7717 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:14.361881 7717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:14.362054 7717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:49:14.363840 7717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "558c4ecd0ece460d9591aa7a3c799d19"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:14.364315 7717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "558c4ecd0ece460d9591aa7a3c799d19"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:14.367673 7717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240426 09:49:14.369827 7731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:14.370949 7717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240426 09:49:14.371052 7717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "558c4ecd0ece460d9591aa7a3c799d19"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:14.371140 7717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:14.393317 7717 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:14.393949 7717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:14.394122 7717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:14.394644 7717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:14.395563 7717 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:14.395612 7717 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:14.395665 7717 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:14.395697 7717 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:14.404302 7717 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:45059
I20240426 09:49:14.404332 7844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:45059 every 8 connection(s)
I20240426 09:49:14.405159 7717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240426 09:49:14.406752 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 7717
I20240426 09:49:14.406888 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240426 09:49:14.413292 7845 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:14.413507 7845 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:14.413910 7845 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:14.414800 6740 ts_manager.cc:194] Registered new tserver with Master: 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:14.415320 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:59135
I20240426 09:49:14.420660 398 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240426 09:49:14.429180 398 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240426 09:49:14.429256 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 7324
I20240426 09:49:14.434661 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 7455
I20240426 09:49:14.439821 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 7586
I20240426 09:49:14.444979 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 7717
I20240426 09:49:14.516984 7190 heartbeater.cc:503] Master 127.0.99.190:43685 was elected leader, sending a full tablet report...
I20240426 09:49:14.689025 7321 heartbeater.cc:503] Master 127.0.99.190:43685 was elected leader, sending a full tablet report...
I20240426 09:49:18.202531 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:18.222043 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 16fb5fe3061842e199528f94fce0f378 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.222044 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 16fb5fe3061842e199528f94fce0f378 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.222498 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 16fb5fe3061842e199528f94fce0f378 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.222874 6992 tablet_service.cc:1467] Processing CreateTablet for tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.223065 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16fb5fe3061842e199528f94fce0f378. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.223065 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16fb5fe3061842e199528f94fce0f378. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.222498 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 43d2cce5794647129a4533a2fb9da89f (DEFAULT_TABLE table=rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.223400 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16fb5fe3061842e199528f94fce0f378. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.223424 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.224128 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43d2cce5794647129a4533a2fb9da89f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.225901 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.226142 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.228658 7124 tablet_service.cc:1467] Processing CreateTablet for tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.229563 7124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.230413 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 43d2cce5794647129a4533a2fb9da89f (DEFAULT_TABLE table=rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.230659 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43d2cce5794647129a4533a2fb9da89f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.231921 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 43d2cce5794647129a4533a2fb9da89f (DEFAULT_TABLE table=rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.232177 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43d2cce5794647129a4533a2fb9da89f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.232306 7866 tablet_bootstrap.cc:492] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:18.233156 7865 tablet_bootstrap.cc:492] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:18.233392 7867 tablet_bootstrap.cc:492] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:18.234750 7866 tablet_bootstrap.cc:654] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.235246 7865 tablet_bootstrap.cc:654] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.235695 7866 log.cc:824] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:18.236033 7865 log.cc:824] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:18.236192 7867 tablet_bootstrap.cc:654] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.236940 7867 log.cc:824] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:18.237820 7865 tablet_bootstrap.cc:492] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:18.237913 7870 tablet_bootstrap.cc:492] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:18.237990 7865 ts_tablet_manager.cc:1367] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:49:18.239280 7866 tablet_bootstrap.cc:492] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:18.239439 7866 ts_tablet_manager.cc:1367] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20240426 09:49:18.239763 7867 tablet_bootstrap.cc:492] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:18.239923 7867 ts_tablet_manager.cc:1367] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20240426 09:49:18.240969 7870 tablet_bootstrap.cc:654] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.241781 7870 log.cc:824] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:18.241919 7865 raft_consensus.cc:348] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:18.242226 7865 raft_consensus.cc:374] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.242307 7865 raft_consensus.cc:729] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.242810 7865 consensus_queue.cc:260] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:18.243026 7866 raft_consensus.cc:348] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.243237 7866 raft_consensus.cc:374] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.243297 7866 raft_consensus.cc:729] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.243695 7867 raft_consensus.cc:348] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.243743 7866 consensus_queue.cc:260] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.243976 7867 raft_consensus.cc:374] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.244068 7867 raft_consensus.cc:729] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.244554 7866 ts_tablet_manager.cc:1398] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240426 09:49:18.244505 7867 consensus_queue.cc:260] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.244725 7865 ts_tablet_manager.cc:1398] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240426 09:49:18.244992 7866 tablet_bootstrap.cc:492] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:18.245011 7865 tablet_bootstrap.cc:492] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:18.245034 7870 tablet_bootstrap.cc:492] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:18.245203 7870 ts_tablet_manager.cc:1367] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20240426 09:49:18.245967 7865 tablet_bootstrap.cc:654] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.246088 7866 tablet_bootstrap.cc:654] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.247128 7865 tablet_bootstrap.cc:492] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:18.247222 7865 ts_tablet_manager.cc:1367] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:18.247604 7867 ts_tablet_manager.cc:1398] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240426 09:49:18.247768 7865 raft_consensus.cc:348] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:18.247861 7865 raft_consensus.cc:374] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.247891 7865 raft_consensus.cc:729] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.247992 7865 consensus_queue.cc:260] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:18.248250 7865 ts_tablet_manager.cc:1398] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:18.248891 7867 tablet_bootstrap.cc:492] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:18.249323 7870 raft_consensus.cc:348] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:18.249562 7870 raft_consensus.cc:374] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.249648 7870 raft_consensus.cc:729] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.249941 7867 tablet_bootstrap.cc:654] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.250133 7870 consensus_queue.cc:260] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:18.251317 7870 ts_tablet_manager.cc:1398] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240426 09:49:18.251380 7866 tablet_bootstrap.cc:492] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:18.251462 7866 ts_tablet_manager.cc:1367] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:18.251693 7870 tablet_bootstrap.cc:492] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:18.252025 7866 raft_consensus.cc:348] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:18.252136 7866 raft_consensus.cc:374] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.252164 7866 raft_consensus.cc:729] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.252277 7866 consensus_queue.cc:260] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:18.252575 7866 ts_tablet_manager.cc:1398] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:18.252635 7870 tablet_bootstrap.cc:654] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.252743 7866 tablet_bootstrap.cc:492] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:18.253659 7866 tablet_bootstrap.cc:654] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.254061 7867 tablet_bootstrap.cc:492] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:18.254141 7867 ts_tablet_manager.cc:1367] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:18.254626 7870 tablet_bootstrap.cc:492] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:18.254758 7870 ts_tablet_manager.cc:1367] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:18.254730 7867 raft_consensus.cc:348] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:18.254904 7867 raft_consensus.cc:374] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.254967 7867 raft_consensus.cc:729] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.255133 7867 consensus_queue.cc:260] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:18.255435 7867 ts_tablet_manager.cc:1398] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:18.255754 7866 tablet_bootstrap.cc:492] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:18.255548 7870 raft_consensus.cc:348] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.255884 7870 raft_consensus.cc:374] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.255940 7870 raft_consensus.cc:729] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.256027 7866 ts_tablet_manager.cc:1367] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:18.256073 7870 consensus_queue.cc:260] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.256353 7870 ts_tablet_manager.cc:1398] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:18.256559 7866 raft_consensus.cc:348] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:18.256731 7866 raft_consensus.cc:374] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.256919 7866 raft_consensus.cc:729] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.257046 7866 consensus_queue.cc:260] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:18.257351 7866 ts_tablet_manager.cc:1398] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240426 09:49:18.257618 7191 tablet.cc:2369] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240426 09:49:18.322896 7060 tablet.cc:2369] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:18.344668 7874 raft_consensus.cc:482] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:18.344871 7874 raft_consensus.cc:504] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:18.346014 7874 leader_election.cc:288] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:18.349865 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16fb5fe3061842e199528f94fce0f378" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:18.349893 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16fb5fe3061842e199528f94fce0f378" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:18.350147 7013 raft_consensus.cc:2454] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:18.350147 7275 raft_consensus.cc:2454] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:18.350579 6815 leader_election.cc:302] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:18.350862 7874 raft_consensus.cc:2781] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:18.350932 7874 raft_consensus.cc:482] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:18.350977 7874 raft_consensus.cc:3037] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.351845 7874 raft_consensus.cc:504] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:18.352175 7874 leader_election.cc:288] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:18.352502 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16fb5fe3061842e199528f94fce0f378" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:18.352511 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16fb5fe3061842e199528f94fce0f378" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:18.352602 7275 raft_consensus.cc:3037] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.352617 7013 raft_consensus.cc:3037] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.353451 7275 raft_consensus.cc:2454] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:18.353452 7013 raft_consensus.cc:2454] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:18.353731 6815 leader_election.cc:302] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:18.353880 7874 raft_consensus.cc:2781] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:18.354092 7874 raft_consensus.cc:686] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:18.354404 7874 consensus_queue.cc:237] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:18.357259 6739 catalog_manager.cc:5484] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:18.389046 7879 raft_consensus.cc:482] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:18.389257 7879 raft_consensus.cc:504] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.390343 7879 leader_election.cc:288] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:18.394042 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d764ebe4e3d4b958d77bcbbc9f8f8d2" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:18.394197 7013 raft_consensus.cc:2454] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:18.394138 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d764ebe4e3d4b958d77bcbbc9f8f8d2" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:18.394373 6882 raft_consensus.cc:2454] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:18.394567 7077 leader_election.cc:302] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:18.394883 7879 raft_consensus.cc:2781] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:18.394958 7879 raft_consensus.cc:482] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:18.395005 7879 raft_consensus.cc:3037] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.395879 7879 raft_consensus.cc:504] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.396216 7879 leader_election.cc:288] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:18.396570 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d764ebe4e3d4b958d77bcbbc9f8f8d2" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:18.396596 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d764ebe4e3d4b958d77bcbbc9f8f8d2" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:18.396693 6882 raft_consensus.cc:3037] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.396708 7013 raft_consensus.cc:3037] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.397473 7013 raft_consensus.cc:2454] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:18.397559 6882 raft_consensus.cc:2454] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:18.397732 7077 leader_election.cc:302] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:18.397934 7879 raft_consensus.cc:2781] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:18.398166 7879 raft_consensus.cc:686] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:18.398470 7879 consensus_queue.cc:237] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
W20240426 09:49:18.400125 6929 tablet.cc:2369] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:18.401319 6739 catalog_manager.cc:5484] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
W20240426 09:49:18.429916 7322 tablet.cc:2369] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:18.529932 7874 raft_consensus.cc:482] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:18.530090 7874 raft_consensus.cc:504] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:18.530570 7874 leader_election.cc:288] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:18.531095 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43d2cce5794647129a4533a2fb9da89f" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:18.531242 7275 raft_consensus.cc:2454] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:18.531569 6817 leader_election.cc:302] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:18.531796 7874 raft_consensus.cc:2781] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:18.531874 7874 raft_consensus.cc:482] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:18.531927 7874 raft_consensus.cc:3037] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.532745 7874 raft_consensus.cc:504] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:18.533083 7874 leader_election.cc:288] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:18.533447 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43d2cce5794647129a4533a2fb9da89f" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:18.533558 7275 raft_consensus.cc:3037] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.534340 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43d2cce5794647129a4533a2fb9da89f" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:18.534483 7275 raft_consensus.cc:2454] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:18.534557 7143 raft_consensus.cc:3037] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.534811 6817 leader_election.cc:302] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:18.535025 7874 raft_consensus.cc:2781] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:18.535094 7874 raft_consensus.cc:686] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:18.535231 7874 consensus_queue.cc:237] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:18.535624 7143 raft_consensus.cc:2454] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:18.534340 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43d2cce5794647129a4533a2fb9da89f" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:18.536278 7144 raft_consensus.cc:2362] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1. Re-sending same reply.
I20240426 09:49:18.536902 6739 catalog_manager.cc:5484] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
W20240426 09:49:18.683534 7891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:18.683809 7891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:18.686803 7891 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240426 09:49:18.785681 7275 raft_consensus.cc:1261] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:18.785791 7274 raft_consensus.cc:1261] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:18.786504 7144 raft_consensus.cc:1261] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:18.787060 7874 consensus_queue.cc:1035] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:18.787249 7013 raft_consensus.cc:1261] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:18.787770 7884 consensus_queue.cc:1035] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:18.787861 7874 consensus_queue.cc:1035] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:18.790709 7884 consensus_queue.cc:1035] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:18.793040 6882 raft_consensus.cc:1261] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:18.793226 7012 raft_consensus.cc:1261] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:18.794332 7888 consensus_queue.cc:1035] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:18.794579 7921 mvcc.cc:205] Tried to move back new op lower bound from 7021055831178760192 to 7021055830161522688. Current Snapshot: MvccSnapshot[applied={T|T < 7021055831178760192}]
I20240426 09:49:18.794709 7888 consensus_queue.cc:1035] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:18.800616 7924 mvcc.cc:205] Tried to move back new op lower bound from 7021055831178760192 to 7021055830161522688. Current Snapshot: MvccSnapshot[applied={T|T < 7021055831178760192}]
I20240426 09:49:18.804216 7930 mvcc.cc:205] Tried to move back new op lower bound from 7021055831178760192 to 7021055830161522688. Current Snapshot: MvccSnapshot[applied={T|T < 7021055831178760192}]
I20240426 09:49:18.830935 7934 mvcc.cc:205] Tried to move back new op lower bound from 7021055831210127360 to 7021055829602656256. Current Snapshot: MvccSnapshot[applied={T|T < 7021055831210127360}]
INSERT report
rows total: 2000
time total: 126.022 ms
time per row: 0.0630111 ms
I20240426 09:49:18.861203 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:18.868459 7124 tablet_service.cc:1467] Processing CreateTablet for tablet b2d42c52b7a94e32ab4cdad78ce884ee (DEFAULT_TABLE table=rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.868796 7124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d42c52b7a94e32ab4cdad78ce884ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.868783 7255 tablet_service.cc:1467] Processing CreateTablet for tablet bd077eafe82d4531a6850e6686a7e566 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.868788 7254 tablet_service.cc:1467] Processing CreateTablet for tablet b2d42c52b7a94e32ab4cdad78ce884ee (DEFAULT_TABLE table=rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.869071 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd077eafe82d4531a6850e6686a7e566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.869242 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d42c52b7a94e32ab4cdad78ce884ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.869542 7123 tablet_service.cc:1467] Processing CreateTablet for tablet bd077eafe82d4531a6850e6686a7e566 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.869755 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd077eafe82d4531a6850e6686a7e566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.869843 6862 tablet_service.cc:1467] Processing CreateTablet for tablet aca60cffb8cf41e3b0d9b64436c306d7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.870126 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca60cffb8cf41e3b0d9b64436c306d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.871454 6992 tablet_service.cc:1467] Processing CreateTablet for tablet bd077eafe82d4531a6850e6686a7e566 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.871619 7953 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:18.871702 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd077eafe82d4531a6850e6686a7e566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.871963 7122 tablet_service.cc:1467] Processing CreateTablet for tablet aca60cffb8cf41e3b0d9b64436c306d7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.872182 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca60cffb8cf41e3b0d9b64436c306d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.872517 7953 tablet_bootstrap.cc:654] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.872485 6993 tablet_service.cc:1467] Processing CreateTablet for tablet b2d42c52b7a94e32ab4cdad78ce884ee (DEFAULT_TABLE table=rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.872730 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d42c52b7a94e32ab4cdad78ce884ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.873922 7957 tablet_bootstrap.cc:492] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:18.874053 6991 tablet_service.cc:1467] Processing CreateTablet for tablet aca60cffb8cf41e3b0d9b64436c306d7 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:18.874275 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca60cffb8cf41e3b0d9b64436c306d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:18.874735 7957 tablet_bootstrap.cc:654] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.876695 7955 tablet_bootstrap.cc:492] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:18.876749 7957 tablet_bootstrap.cc:492] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:18.876839 7957 ts_tablet_manager.cc:1367] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:18.877269 7957 raft_consensus.cc:348] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.877373 7957 raft_consensus.cc:374] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.877404 7957 raft_consensus.cc:729] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.877532 7957 consensus_queue.cc:260] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.877617 7955 tablet_bootstrap.cc:654] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.877967 7957 ts_tablet_manager.cc:1398] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:18.878144 7957 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:18.878238 7954 tablet_bootstrap.cc:492] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:18.879215 7957 tablet_bootstrap.cc:654] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.879552 7954 tablet_bootstrap.cc:654] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.880736 7955 tablet_bootstrap.cc:492] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:18.880824 7955 ts_tablet_manager.cc:1367] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:18.881276 7955 raft_consensus.cc:348] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.881388 7955 raft_consensus.cc:374] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.881420 7955 raft_consensus.cc:729] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.881580 7955 consensus_queue.cc:260] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.881899 7955 ts_tablet_manager.cc:1398] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:18.882387 7954 tablet_bootstrap.cc:492] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:18.882465 7954 ts_tablet_manager.cc:1367] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:49:18.882966 7954 raft_consensus.cc:348] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.883072 7954 raft_consensus.cc:374] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.883111 7954 raft_consensus.cc:729] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.883358 7953 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:18.883445 7953 ts_tablet_manager.cc:1367] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:49:18.883924 7953 raft_consensus.cc:348] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.884039 7953 raft_consensus.cc:374] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.884069 7953 raft_consensus.cc:729] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.884191 7953 consensus_queue.cc:260] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.883252 7954 consensus_queue.cc:260] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.884462 7953 ts_tablet_manager.cc:1398] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:18.884639 7953 tablet_bootstrap.cc:492] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:18.884765 7954 ts_tablet_manager.cc:1398] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:18.884968 7954 tablet_bootstrap.cc:492] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:18.885560 7953 tablet_bootstrap.cc:654] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.885902 7954 tablet_bootstrap.cc:654] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.887002 7957 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:18.887090 7957 ts_tablet_manager.cc:1367] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:18.887565 7957 raft_consensus.cc:348] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.887682 7957 raft_consensus.cc:374] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.887717 7957 raft_consensus.cc:729] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.887881 7957 consensus_queue.cc:260] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.888257 7957 ts_tablet_manager.cc:1398] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:18.888424 7957 tablet_bootstrap.cc:492] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:18.889614 7957 tablet_bootstrap.cc:654] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.890504 7954 tablet_bootstrap.cc:492] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:18.890596 7954 ts_tablet_manager.cc:1367] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240426 09:49:18.890388 7953 tablet_bootstrap.cc:492] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:18.890982 7953 ts_tablet_manager.cc:1367] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:18.891151 7957 tablet_bootstrap.cc:492] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:18.891232 7957 ts_tablet_manager.cc:1367] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:18.891587 7953 raft_consensus.cc:348] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.891630 7957 raft_consensus.cc:348] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.891716 7957 raft_consensus.cc:374] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.891752 7957 raft_consensus.cc:729] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.891834 7953 raft_consensus.cc:374] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.891920 7953 raft_consensus.cc:729] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.891911 7957 consensus_queue.cc:260] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.892131 7953 consensus_queue.cc:260] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.892194 7957 ts_tablet_manager.cc:1398] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:18.892576 7954 raft_consensus.cc:348] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.892693 7954 raft_consensus.cc:374] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.892729 7954 raft_consensus.cc:729] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.892875 7954 consensus_queue.cc:260] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.893236 7954 ts_tablet_manager.cc:1398] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:18.893406 7954 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:18.894246 7953 ts_tablet_manager.cc:1398] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:18.894327 7954 tablet_bootstrap.cc:654] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:18.895936 7954 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:18.896020 7954 ts_tablet_manager.cc:1367] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:18.896445 7954 raft_consensus.cc:348] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.896544 7954 raft_consensus.cc:374] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:18.896577 7954 raft_consensus.cc:729] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:18.896710 7954 consensus_queue.cc:260] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.897074 7954 ts_tablet_manager.cc:1398] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:18.936107 7879 raft_consensus.cc:482] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:18.936223 7879 raft_consensus.cc:504] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.936627 7879 leader_election.cc:288] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:18.937064 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca60cffb8cf41e3b0d9b64436c306d7" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:18.937064 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca60cffb8cf41e3b0d9b64436c306d7" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:18.937215 7012 raft_consensus.cc:2454] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:18.937216 6882 raft_consensus.cc:2454] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:18.937530 7077 leader_election.cc:302] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:18.937719 7879 raft_consensus.cc:2781] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:18.937788 7879 raft_consensus.cc:482] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:18.937831 7879 raft_consensus.cc:3037] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.938706 7879 raft_consensus.cc:504] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.939067 7879 leader_election.cc:288] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:18.939354 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca60cffb8cf41e3b0d9b64436c306d7" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:18.939450 7012 raft_consensus.cc:3037] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.939404 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca60cffb8cf41e3b0d9b64436c306d7" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:18.939532 6882 raft_consensus.cc:3037] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:18.940244 6882 raft_consensus.cc:2454] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:18.940249 7012 raft_consensus.cc:2454] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:18.940495 7077 leader_election.cc:302] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:18.940681 7879 raft_consensus.cc:2781] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:18.940757 7879 raft_consensus.cc:686] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:18.940892 7879 consensus_queue.cc:237] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:18.942344 6739 catalog_manager.cc:5484] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:19.000800 7923 raft_consensus.cc:482] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:19.000977 7923 raft_consensus.cc:504] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.002012 7923 leader_election.cc:288] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:19.006649 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d42c52b7a94e32ab4cdad78ce884ee" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:19.006843 7012 raft_consensus.cc:2454] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:19.007093 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d42c52b7a94e32ab4cdad78ce884ee" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:19.007182 7208 leader_election.cc:302] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:19.007236 7144 raft_consensus.cc:2454] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:19.007433 7923 raft_consensus.cc:2781] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:19.007498 7923 raft_consensus.cc:482] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:19.007544 7923 raft_consensus.cc:3037] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.008448 7923 raft_consensus.cc:504] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.008783 7923 leader_election.cc:288] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:19.009102 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d42c52b7a94e32ab4cdad78ce884ee" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:19.009132 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d42c52b7a94e32ab4cdad78ce884ee" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:19.009222 7144 raft_consensus.cc:3037] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.009223 7012 raft_consensus.cc:3037] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.010159 7012 raft_consensus.cc:2454] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:19.010170 7144 raft_consensus.cc:2454] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:19.010427 7208 leader_election.cc:302] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:19.010567 7923 raft_consensus.cc:2781] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:19.010741 7923 raft_consensus.cc:686] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:19.011114 7923 consensus_queue.cc:237] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.013374 6739 catalog_manager.cc:5484] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:19.143991 7933 raft_consensus.cc:482] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:19.144131 7933 raft_consensus.cc:504] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.145218 7933 leader_election.cc:288] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:19.149749 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd077eafe82d4531a6850e6686a7e566" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:19.149928 7275 raft_consensus.cc:2454] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:19.149943 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd077eafe82d4531a6850e6686a7e566" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:19.150100 7144 raft_consensus.cc:2454] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:19.150444 6948 leader_election.cc:302] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:19.150779 7933 raft_consensus.cc:2781] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:19.150856 7933 raft_consensus.cc:482] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:19.150897 7933 raft_consensus.cc:3037] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.151938 7933 raft_consensus.cc:504] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.152307 7933 leader_election.cc:288] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:19.152630 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd077eafe82d4531a6850e6686a7e566" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:19.152642 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd077eafe82d4531a6850e6686a7e566" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:19.152745 7144 raft_consensus.cc:3037] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.152751 7275 raft_consensus.cc:3037] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.153507 7144 raft_consensus.cc:2454] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:19.153647 7275 raft_consensus.cc:2454] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:19.153833 6946 leader_election.cc:302] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:19.154006 7933 raft_consensus.cc:2781] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:19.154117 7933 raft_consensus.cc:686] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:19.154414 7933 consensus_queue.cc:237] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.156992 6739 catalog_manager.cc:5484] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
W20240426 09:49:19.282403 7976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:19.282682 7976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:19.285776 7976 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240426 09:49:19.361053 7012 raft_consensus.cc:1261] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.361585 7879 consensus_queue.cc:1035] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.363000 7013 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.363301 7144 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.363791 7275 raft_consensus.cc:1261] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.364084 7144 raft_consensus.cc:1261] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.364579 7929 consensus_queue.cc:1035] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.364946 7929 consensus_queue.cc:1035] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.365177 7923 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.365778 7923 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.376009 6882 raft_consensus.cc:1261] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.376504 7879 consensus_queue.cc:1035] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 103.757 ms
time per row: 0.0518784 ms
I20240426 09:49:19.430038 6731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:19.435526 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 565d9e485cf144de8c0018f64aacb8b4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.435617 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 565d9e485cf144de8c0018f64aacb8b4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.435860 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565d9e485cf144de8c0018f64aacb8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.435917 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565d9e485cf144de8c0018f64aacb8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.436353 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 8786902a28ee47a1995f77e58e18aa5d (DEFAULT_TABLE table=rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.436568 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8786902a28ee47a1995f77e58e18aa5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.437999 8033 tablet_bootstrap.cc:492] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:19.438453 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 565d9e485cf144de8c0018f64aacb8b4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.438462 7124 tablet_service.cc:1467] Processing CreateTablet for tablet 72e2daf3e0d84a12bad5ff76fdd01c88 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.438721 7124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e2daf3e0d84a12bad5ff76fdd01c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.438735 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565d9e485cf144de8c0018f64aacb8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.439119 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 8786902a28ee47a1995f77e58e18aa5d (DEFAULT_TABLE table=rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.439179 8033 tablet_bootstrap.cc:654] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.439323 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8786902a28ee47a1995f77e58e18aa5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.440622 8036 tablet_bootstrap.cc:492] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:19.441365 8033 tablet_bootstrap.cc:492] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:19.441385 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 72e2daf3e0d84a12bad5ff76fdd01c88 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.441459 8033 ts_tablet_manager.cc:1367] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:19.441541 8037 tablet_bootstrap.cc:492] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:19.441587 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e2daf3e0d84a12bad5ff76fdd01c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.442026 8033 raft_consensus.cc:348] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:19.442134 8033 raft_consensus.cc:374] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.442163 8033 raft_consensus.cc:729] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.442308 8033 consensus_queue.cc:260] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:19.442459 8037 tablet_bootstrap.cc:654] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.442720 8033 ts_tablet_manager.cc:1398] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.442965 8033 tablet_bootstrap.cc:492] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:19.443737 8037 tablet_bootstrap.cc:492] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:19.443827 8037 ts_tablet_manager.cc:1367] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:19.444036 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 72e2daf3e0d84a12bad5ff76fdd01c88 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.444267 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72e2daf3e0d84a12bad5ff76fdd01c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.444537 8034 tablet_bootstrap.cc:492] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:19.444691 8033 tablet_bootstrap.cc:654] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.445410 8034 tablet_bootstrap.cc:654] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.445700 8036 tablet_bootstrap.cc:654] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.445870 8037 raft_consensus.cc:348] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.445971 8037 raft_consensus.cc:374] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.446000 8037 raft_consensus.cc:729] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.446146 8037 consensus_queue.cc:260] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.446408 8037 ts_tablet_manager.cc:1398] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:19.446368 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 8786902a28ee47a1995f77e58e18aa5d (DEFAULT_TABLE table=rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.446571 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8786902a28ee47a1995f77e58e18aa5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.447198 8034 tablet_bootstrap.cc:492] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:19.447271 8034 ts_tablet_manager.cc:1367] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:19.447700 8034 raft_consensus.cc:348] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:19.447813 8034 raft_consensus.cc:374] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.447844 8034 raft_consensus.cc:729] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.447973 8034 consensus_queue.cc:260] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:19.448031 8033 tablet_bootstrap.cc:492] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:19.448101 8033 ts_tablet_manager.cc:1367] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:19.448408 8034 ts_tablet_manager.cc:1398] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.448560 8034 tablet_bootstrap.cc:492] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:19.449349 8034 tablet_bootstrap.cc:654] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.450062 8033 raft_consensus.cc:348] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.450186 8033 raft_consensus.cc:374] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.450222 8033 raft_consensus.cc:729] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.450258 8036 tablet_bootstrap.cc:492] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:19.450356 8036 ts_tablet_manager.cc:1367] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240426 09:49:19.450516 8034 tablet_bootstrap.cc:492] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:19.450590 8034 ts_tablet_manager.cc:1367] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:19.450856 8036 raft_consensus.cc:348] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:19.450963 8036 raft_consensus.cc:374] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.450995 8036 raft_consensus.cc:729] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.450992 8033 consensus_queue.cc:260] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.451016 8034 raft_consensus.cc:348] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.451109 8034 raft_consensus.cc:374] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.451153 8034 raft_consensus.cc:729] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.451141 8036 consensus_queue.cc:260] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:19.451265 8034 consensus_queue.cc:260] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.451522 8036 ts_tablet_manager.cc:1398] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.451532 8034 ts_tablet_manager.cc:1398] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:19.451697 8036 tablet_bootstrap.cc:492] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:19.451709 8034 tablet_bootstrap.cc:492] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:19.452638 8036 tablet_bootstrap.cc:654] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.452641 8034 tablet_bootstrap.cc:654] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.453328 8033 ts_tablet_manager.cc:1398] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:19.454047 8034 tablet_bootstrap.cc:492] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:19.454144 8034 ts_tablet_manager.cc:1367] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:19.454550 8034 raft_consensus.cc:348] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.454634 8034 raft_consensus.cc:374] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.454666 8034 raft_consensus.cc:729] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.454813 8034 consensus_queue.cc:260] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.455127 8034 ts_tablet_manager.cc:1398] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.455242 8036 tablet_bootstrap.cc:492] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:19.455313 8036 ts_tablet_manager.cc:1367] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:19.455701 8036 raft_consensus.cc:348] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.455817 8036 raft_consensus.cc:374] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.455859 8036 raft_consensus.cc:729] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.456012 8036 consensus_queue.cc:260] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.456302 8036 ts_tablet_manager.cc:1398] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:19.456503 8036 tablet_bootstrap.cc:492] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:19.458012 8036 tablet_bootstrap.cc:654] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.459035 8036 tablet_bootstrap.cc:492] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:19.459102 8036 ts_tablet_manager.cc:1367] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:49:19.459401 8036 raft_consensus.cc:348] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.459477 8036 raft_consensus.cc:374] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.459506 8036 raft_consensus.cc:729] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.459599 8036 consensus_queue.cc:260] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.459839 8036 ts_tablet_manager.cc:1398] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:19.573379 7970 raft_consensus.cc:482] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:19.573515 7970 raft_consensus.cc:504] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.574021 7970 leader_election.cc:288] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:19.574554 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e2daf3e0d84a12bad5ff76fdd01c88" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:19.574729 7144 raft_consensus.cc:2454] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:19.575145 7208 leader_election.cc:302] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:19.575330 7970 raft_consensus.cc:2781] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:19.575382 7970 raft_consensus.cc:482] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:19.575423 7970 raft_consensus.cc:3037] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.576306 7970 raft_consensus.cc:504] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.576691 7970 leader_election.cc:288] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:19.577006 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e2daf3e0d84a12bad5ff76fdd01c88" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:19.577178 7144 raft_consensus.cc:3037] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.578126 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e2daf3e0d84a12bad5ff76fdd01c88" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:19.578162 7144 raft_consensus.cc:2454] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:19.578182 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72e2daf3e0d84a12bad5ff76fdd01c88" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:19.578261 6882 raft_consensus.cc:2454] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:19.578462 7208 leader_election.cc:302] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:19.578655 7970 raft_consensus.cc:2781] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:19.578744 7970 raft_consensus.cc:686] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:19.578924 7970 consensus_queue.cc:237] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.580479 6738 catalog_manager.cc:5484] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:19.601727 7933 raft_consensus.cc:482] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:19.601871 7933 raft_consensus.cc:504] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:19.602319 7933 leader_election.cc:288] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:19.603024 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565d9e485cf144de8c0018f64aacb8b4" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:19.603166 7275 raft_consensus.cc:2454] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:19.603467 6948 leader_election.cc:302] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:19.603646 7933 raft_consensus.cc:2781] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:19.603691 7933 raft_consensus.cc:482] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:19.603736 7933 raft_consensus.cc:3037] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.604506 7933 raft_consensus.cc:504] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:19.604832 7933 leader_election.cc:288] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:19.605150 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565d9e485cf144de8c0018f64aacb8b4" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:19.605255 7275 raft_consensus.cc:3037] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.605697 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565d9e485cf144de8c0018f64aacb8b4" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:19.605697 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565d9e485cf144de8c0018f64aacb8b4" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:19.605885 6881 raft_consensus.cc:2454] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:19.606211 7275 raft_consensus.cc:2454] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:19.606483 6948 leader_election.cc:302] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters: c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:49:19.606645 7933 raft_consensus.cc:2781] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:19.606725 7933 raft_consensus.cc:686] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:19.606899 7933 consensus_queue.cc:237] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:19.608476 6738 catalog_manager.cc:5484] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:19.665620 7933 raft_consensus.cc:482] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:19.665774 7933 raft_consensus.cc:504] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.666194 7933 leader_election.cc:288] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:19.666725 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8786902a28ee47a1995f77e58e18aa5d" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:19.666750 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8786902a28ee47a1995f77e58e18aa5d" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:19.666903 6882 raft_consensus.cc:2454] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:19.666956 7275 raft_consensus.cc:2454] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:19.667237 6948 leader_election.cc:302] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:19.667418 7933 raft_consensus.cc:2781] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:19.667476 7933 raft_consensus.cc:482] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:19.667520 7933 raft_consensus.cc:3037] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.668313 7933 raft_consensus.cc:504] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.668664 7933 leader_election.cc:288] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:19.669059 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8786902a28ee47a1995f77e58e18aa5d" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:19.669059 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8786902a28ee47a1995f77e58e18aa5d" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:19.669170 7275 raft_consensus.cc:3037] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.669170 6882 raft_consensus.cc:3037] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.670083 6882 raft_consensus.cc:2454] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:19.670083 7275 raft_consensus.cc:2454] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:19.670382 6948 leader_election.cc:302] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:19.670547 7933 raft_consensus.cc:2781] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:19.670650 7933 raft_consensus.cc:686] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:19.670814 7933 consensus_queue.cc:237] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:19.672415 6738 catalog_manager.cc:5484] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
W20240426 09:49:19.784708 8051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:19.784946 8051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:19.787824 8051 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240426 09:49:19.876866 7275 raft_consensus.cc:1261] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.877540 6882 raft_consensus.cc:3037] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.877542 7933 consensus_queue.cc:1035] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.878634 6882 raft_consensus.cc:1261] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.879770 7275 raft_consensus.cc:1261] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.880038 7933 consensus_queue.cc:1035] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.881009 7144 raft_consensus.cc:1261] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.881415 7929 consensus_queue.cc:1035] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.881867 7923 consensus_queue.cc:1035] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.883950 6880 raft_consensus.cc:1261] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.884310 7929 consensus_queue.cc:1035] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:19.885185 6882 raft_consensus.cc:3037] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.886152 6882 raft_consensus.cc:1261] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:19.886627 7970 consensus_queue.cc:1035] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 106.81 ms
time per row: 0.0534048 ms
I20240426 09:49:19.947686 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:19.954536 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 639d1d6189fb4d5f8a2d0aa0d39575ec (DEFAULT_TABLE table=rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.955293 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 639d1d6189fb4d5f8a2d0aa0d39575ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.957021 7124 tablet_service.cc:1467] Processing CreateTablet for tablet 639d1d6189fb4d5f8a2d0aa0d39575ec (DEFAULT_TABLE table=rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.957320 7124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 639d1d6189fb4d5f8a2d0aa0d39575ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.958319 8092 tablet_bootstrap.cc:492] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:19.959118 7122 tablet_service.cc:1467] Processing CreateTablet for tablet a8912e37d7e941a791dd77560e2b3b6b (DEFAULT_TABLE table=rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.959118 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 639d1d6189fb4d5f8a2d0aa0d39575ec (DEFAULT_TABLE table=rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.959266 8092 tablet_bootstrap.cc:654] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.959348 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 639d1d6189fb4d5f8a2d0aa0d39575ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.959348 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8912e37d7e941a791dd77560e2b3b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.959429 7254 tablet_service.cc:1467] Processing CreateTablet for tablet a8912e37d7e941a791dd77560e2b3b6b (DEFAULT_TABLE table=rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.959718 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8912e37d7e941a791dd77560e2b3b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.961334 8095 tablet_bootstrap.cc:492] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:19.961503 6862 tablet_service.cc:1467] Processing CreateTablet for tablet a8912e37d7e941a791dd77560e2b3b6b (DEFAULT_TABLE table=rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.961737 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8912e37d7e941a791dd77560e2b3b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.962240 8095 tablet_bootstrap.cc:654] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.962604 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 9c94c820ab65420a8d3f885ccc9f8884 (DEFAULT_TABLE table=rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.962862 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c94c820ab65420a8d3f885ccc9f8884. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.962879 8094 tablet_bootstrap.cc:492] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:19.963627 8094 tablet_bootstrap.cc:654] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.963868 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 9c94c820ab65420a8d3f885ccc9f8884 (DEFAULT_TABLE table=rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.964141 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c94c820ab65420a8d3f885ccc9f8884. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.964744 8095 tablet_bootstrap.cc:492] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:19.964876 8094 tablet_bootstrap.cc:492] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:19.964888 8095 ts_tablet_manager.cc:1367] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:19.964965 8094 ts_tablet_manager.cc:1367] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:19.965306 8095 raft_consensus.cc:348] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.965361 8094 raft_consensus.cc:348] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.965430 8095 raft_consensus.cc:374] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.965446 8094 raft_consensus.cc:374] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.965466 8095 raft_consensus.cc:729] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.965490 8094 raft_consensus.cc:729] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.965618 8094 consensus_queue.cc:260] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.965920 8094 ts_tablet_manager.cc:1398] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.966104 8094 tablet_bootstrap.cc:492] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:19.966282 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 9c94c820ab65420a8d3f885ccc9f8884 (DEFAULT_TABLE table=rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:19.966554 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c94c820ab65420a8d3f885ccc9f8884. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:19.965616 8095 consensus_queue.cc:260] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.967082 8094 tablet_bootstrap.cc:654] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.967145 8095 ts_tablet_manager.cc:1398] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:19.967307 8095 tablet_bootstrap.cc:492] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:19.968751 8092 tablet_bootstrap.cc:492] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:19.968838 8092 ts_tablet_manager.cc:1367] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:19.969020 8095 tablet_bootstrap.cc:654] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.969357 8092 raft_consensus.cc:348] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.969458 8092 raft_consensus.cc:374] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.969461 8096 tablet_bootstrap.cc:492] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:19.969491 8092 raft_consensus.cc:729] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.969619 8092 consensus_queue.cc:260] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.969918 8092 ts_tablet_manager.cc:1398] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:19.970048 8092 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:19.970973 8092 tablet_bootstrap.cc:654] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.971242 8096 tablet_bootstrap.cc:654] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.971930 8094 tablet_bootstrap.cc:492] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:19.972023 8094 ts_tablet_manager.cc:1367] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:19.972594 8096 tablet_bootstrap.cc:492] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:19.972537 8094 raft_consensus.cc:348] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.972658 8094 raft_consensus.cc:374] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.972690 8096 ts_tablet_manager.cc:1367] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:19.972697 8094 raft_consensus.cc:729] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.972820 8094 consensus_queue.cc:260] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.973094 8094 ts_tablet_manager.cc:1398] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:19.973151 8096 raft_consensus.cc:348] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.973268 8096 raft_consensus.cc:374] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.973309 8096 raft_consensus.cc:729] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.973433 8096 consensus_queue.cc:260] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.973706 8096 ts_tablet_manager.cc:1398] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.973783 8092 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:19.973855 8092 ts_tablet_manager.cc:1367] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:19.973872 8096 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:19.974436 8095 tablet_bootstrap.cc:492] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:19.974521 8095 ts_tablet_manager.cc:1367] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:19.974506 8092 raft_consensus.cc:348] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:19.974591 8092 raft_consensus.cc:374] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.974624 8092 raft_consensus.cc:729] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.974790 8096 tablet_bootstrap.cc:654] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.974735 8092 consensus_queue.cc:260] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:19.975011 8092 ts_tablet_manager.cc:1398] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.975005 8095 raft_consensus.cc:348] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.975101 8095 raft_consensus.cc:374] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.975136 8095 raft_consensus.cc:729] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.975263 8095 consensus_queue.cc:260] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.975543 8095 ts_tablet_manager.cc:1398] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.975732 8095 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:19.976603 8095 tablet_bootstrap.cc:654] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:19.978214 8095 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:19.978307 8095 ts_tablet_manager.cc:1367] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:19.978390 8096 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:19.978463 8096 ts_tablet_manager.cc:1367] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:49:19.978708 8095 raft_consensus.cc:348] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:19.978842 8095 raft_consensus.cc:374] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.978889 8095 raft_consensus.cc:729] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.978891 8096 raft_consensus.cc:348] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:19.978986 8096 raft_consensus.cc:374] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:19.979038 8096 raft_consensus.cc:729] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:19.979003 8095 consensus_queue.cc:260] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:19.979203 8096 consensus_queue.cc:260] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:19.979290 8095 ts_tablet_manager.cc:1398] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.979471 8096 ts_tablet_manager.cc:1398] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:19.979799 7879 raft_consensus.cc:482] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:19.979887 7879 raft_consensus.cc:504] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.980238 7879 leader_election.cc:288] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:19.980613 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "639d1d6189fb4d5f8a2d0aa0d39575ec" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:19.980649 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "639d1d6189fb4d5f8a2d0aa0d39575ec" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:19.980724 7013 raft_consensus.cc:2454] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:19.980746 6880 raft_consensus.cc:2454] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:19.981036 7079 leader_election.cc:302] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:19.981216 7879 raft_consensus.cc:2781] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:19.981269 7879 raft_consensus.cc:482] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:19.981319 7879 raft_consensus.cc:3037] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.982574 7879 raft_consensus.cc:504] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.982971 7879 leader_election.cc:288] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:19.983201 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "639d1d6189fb4d5f8a2d0aa0d39575ec" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:19.983245 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "639d1d6189fb4d5f8a2d0aa0d39575ec" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:19.983309 7013 raft_consensus.cc:3037] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.983333 6880 raft_consensus.cc:3037] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:19.984277 6880 raft_consensus.cc:2454] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:19.984277 7013 raft_consensus.cc:2454] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:19.984534 7079 leader_election.cc:302] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:19.984684 7879 raft_consensus.cc:2781] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:19.984854 7879 raft_consensus.cc:686] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:19.984992 7879 consensus_queue.cc:237] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:19.986230 6739 catalog_manager.cc:5484] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:20.143821 7929 raft_consensus.cc:482] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:20.143976 7929 raft_consensus.cc:504] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:20.144416 7929 leader_election.cc:288] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:20.144949 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c94c820ab65420a8d3f885ccc9f8884" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:20.145110 7275 raft_consensus.cc:2454] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:20.145346 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c94c820ab65420a8d3f885ccc9f8884" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:20.145493 7144 raft_consensus.cc:2454] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:20.145509 6948 leader_election.cc:302] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:20.145813 7929 raft_consensus.cc:2781] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:20.145874 7929 raft_consensus.cc:482] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:20.145910 7929 raft_consensus.cc:3037] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.146976 7929 raft_consensus.cc:504] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:20.147326 7929 leader_election.cc:288] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:20.147697 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c94c820ab65420a8d3f885ccc9f8884" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:20.147814 7275 raft_consensus.cc:3037] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.148029 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c94c820ab65420a8d3f885ccc9f8884" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:20.148128 7144 raft_consensus.cc:3037] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.148840 7275 raft_consensus.cc:2454] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:20.149148 7144 raft_consensus.cc:2454] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:20.149230 6948 leader_election.cc:302] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:20.149446 7929 raft_consensus.cc:2781] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:20.149521 7929 raft_consensus.cc:686] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:20.149677 7929 consensus_queue.cc:237] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:20.151693 6739 catalog_manager.cc:5484] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:20.227286 7923 raft_consensus.cc:482] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:20.227447 7923 raft_consensus.cc:504] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.227877 7923 leader_election.cc:288] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:20.228344 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8912e37d7e941a791dd77560e2b3b6b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:20.228345 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8912e37d7e941a791dd77560e2b3b6b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:20.228484 6880 raft_consensus.cc:2454] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:20.228494 7144 raft_consensus.cc:2454] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:20.228816 7208 leader_election.cc:302] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:20.228999 7923 raft_consensus.cc:2781] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:20.229076 7923 raft_consensus.cc:482] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:20.229120 7923 raft_consensus.cc:3037] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.229923 7923 raft_consensus.cc:504] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.230266 7923 leader_election.cc:288] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:20.230590 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8912e37d7e941a791dd77560e2b3b6b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:20.230593 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8912e37d7e941a791dd77560e2b3b6b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:20.230695 7144 raft_consensus.cc:3037] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.230706 6880 raft_consensus.cc:3037] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.231420 7144 raft_consensus.cc:2454] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:20.231482 6880 raft_consensus.cc:2454] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:20.231684 7208 leader_election.cc:302] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:20.231865 7923 raft_consensus.cc:2781] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:20.231952 7923 raft_consensus.cc:686] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:20.232086 7923 consensus_queue.cc:237] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.233666 6739 catalog_manager.cc:5484] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
W20240426 09:49:20.365726 8110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:20.365973 8110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:20.368888 8110 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240426 09:49:20.402973 7879 consensus_queue.cc:1035] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:20.407678 8106 consensus_queue.cc:1035] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:20.464386 7144 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:20.464785 7929 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:20.466871 7275 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:20.467525 7929 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:20.475148 6881 raft_consensus.cc:1261] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:20.475207 7143 raft_consensus.cc:1261] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:20.475589 7923 consensus_queue.cc:1035] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:20.476277 7923 consensus_queue.cc:1035] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 125.896 ms
time per row: 0.0629482 ms
I20240426 09:49:20.535827 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:20.541251 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2e6050128b45f3b6866a6770ebafb8 (DEFAULT_TABLE table=rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:20.541271 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2e6050128b45f3b6866a6770ebafb8 (DEFAULT_TABLE table=rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:20.541568 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2e6050128b45f3b6866a6770ebafb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:20.541571 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2e6050128b45f3b6866a6770ebafb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:20.542433 6860 tablet_service.cc:1467] Processing CreateTablet for tablet bcb2d524281d43e4bbb962ed7b71e330 (DEFAULT_TABLE table=rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:20.542454 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 015b439880d94914b0571789407be4f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:20.542713 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb2d524281d43e4bbb962ed7b71e330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:20.542721 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015b439880d94914b0571789407be4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:20.544023 7253 tablet_service.cc:1467] Processing CreateTablet for tablet bcb2d524281d43e4bbb962ed7b71e330 (DEFAULT_TABLE table=rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:20.544027 6991 tablet_service.cc:1467] Processing CreateTablet for tablet bcb2d524281d43e4bbb962ed7b71e330 (DEFAULT_TABLE table=rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:20.544222 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb2d524281d43e4bbb962ed7b71e330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:20.544227 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb2d524281d43e4bbb962ed7b71e330. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:20.546159 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2e6050128b45f3b6866a6770ebafb8 (DEFAULT_TABLE table=rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:20.546308 8173 tablet_bootstrap.cc:492] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:20.546397 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2e6050128b45f3b6866a6770ebafb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:20.546532 8174 tablet_bootstrap.cc:492] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:20.547310 8174 tablet_bootstrap.cc:654] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:20.547322 8173 tablet_bootstrap.cc:654] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:20.548403 8174 tablet_bootstrap.cc:492] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:20.548485 8174 ts_tablet_manager.cc:1367] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:20.548475 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 015b439880d94914b0571789407be4f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:20.548734 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015b439880d94914b0571789407be4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:20.548853 8173 tablet_bootstrap.cc:492] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:20.548941 8173 ts_tablet_manager.cc:1367] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:20.548977 8174 raft_consensus.cc:348] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.549093 8174 raft_consensus.cc:374] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:20.549127 8174 raft_consensus.cc:729] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:20.549260 8174 consensus_queue.cc:260] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.549405 8173 raft_consensus.cc:348] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:20.549507 8174 ts_tablet_manager.cc:1398] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:20.549515 8173 raft_consensus.cc:374] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:20.549554 8173 raft_consensus.cc:729] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:20.549679 8174 tablet_bootstrap.cc:492] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:20.549716 8173 consensus_queue.cc:260] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:20.550104 8173 ts_tablet_manager.cc:1398] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:20.550271 8173 tablet_bootstrap.cc:492] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:20.550465 8174 tablet_bootstrap.cc:654] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:20.551383 8174 tablet_bootstrap.cc:492] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:20.551462 8174 ts_tablet_manager.cc:1367] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:20.551898 8174 raft_consensus.cc:348] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:20.551990 8174 raft_consensus.cc:374] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:20.552018 8174 raft_consensus.cc:729] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:20.552143 8174 consensus_queue.cc:260] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:20.552145 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 015b439880d94914b0571789407be4f9 (DEFAULT_TABLE table=rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:20.552474 8174 ts_tablet_manager.cc:1398] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:20.553074 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015b439880d94914b0571789407be4f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:20.553400 8173 tablet_bootstrap.cc:654] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:20.555171 8173 tablet_bootstrap.cc:492] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:20.555265 8173 ts_tablet_manager.cc:1367] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:20.555667 8172 tablet_bootstrap.cc:492] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:20.555657 8173 raft_consensus.cc:348] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.555752 8173 raft_consensus.cc:374] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:20.555790 8173 raft_consensus.cc:729] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:20.555934 8173 consensus_queue.cc:260] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.556177 8173 ts_tablet_manager.cc:1398] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:20.556505 8172 tablet_bootstrap.cc:654] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:20.556661 8173 tablet_bootstrap.cc:492] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:20.557255 8171 tablet_bootstrap.cc:492] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:20.557425 8173 tablet_bootstrap.cc:654] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:20.558089 8171 tablet_bootstrap.cc:654] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:20.559006 8172 tablet_bootstrap.cc:492] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:20.559096 8172 ts_tablet_manager.cc:1367] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:20.559383 8173 tablet_bootstrap.cc:492] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:20.559465 8173 ts_tablet_manager.cc:1367] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:20.559594 8172 raft_consensus.cc:348] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.559710 8172 raft_consensus.cc:374] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:20.559751 8172 raft_consensus.cc:729] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:20.559876 8172 consensus_queue.cc:260] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.559885 8173 raft_consensus.cc:348] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.559974 8173 raft_consensus.cc:374] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:20.560017 8173 raft_consensus.cc:729] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:20.560137 8173 consensus_queue.cc:260] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.560375 8171 tablet_bootstrap.cc:492] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:20.560408 8173 ts_tablet_manager.cc:1398] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:20.560460 8171 ts_tablet_manager.cc:1367] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:20.560866 8171 raft_consensus.cc:348] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.560972 8171 raft_consensus.cc:374] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:20.561021 8171 raft_consensus.cc:729] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:20.561139 8171 consensus_queue.cc:260] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.561357 8171 ts_tablet_manager.cc:1398] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:20.561511 8171 tablet_bootstrap.cc:492] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:20.562062 8172 ts_tablet_manager.cc:1398] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:49:20.562286 8172 tablet_bootstrap.cc:492] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:20.562757 8171 tablet_bootstrap.cc:654] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:20.563243 8172 tablet_bootstrap.cc:654] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:20.563879 8171 tablet_bootstrap.cc:492] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:20.563959 8171 ts_tablet_manager.cc:1367] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:20.564404 8171 raft_consensus.cc:348] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.564491 8171 raft_consensus.cc:374] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:20.564524 8171 raft_consensus.cc:729] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:20.564641 8171 consensus_queue.cc:260] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.564989 8171 ts_tablet_manager.cc:1398] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:20.565701 8172 tablet_bootstrap.cc:492] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:20.565764 8172 ts_tablet_manager.cc:1367] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240426 09:49:20.566320 8172 raft_consensus.cc:348] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:20.566406 8172 raft_consensus.cc:374] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:20.566437 8172 raft_consensus.cc:729] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:20.566555 8172 consensus_queue.cc:260] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:20.566916 8172 ts_tablet_manager.cc:1398] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240426 09:49:20.624617 8152 raft_consensus.cc:482] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:20.624758 8152 raft_consensus.cc:504] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.625180 8152 leader_election.cc:288] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:20.625571 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2e6050128b45f3b6866a6770ebafb8" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:20.625571 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2e6050128b45f3b6866a6770ebafb8" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:20.625738 7013 raft_consensus.cc:2454] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:20.625738 6880 raft_consensus.cc:2454] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:20.626061 7077 leader_election.cc:302] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:20.626247 8152 raft_consensus.cc:2781] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:20.626315 8152 raft_consensus.cc:482] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:20.626366 8152 raft_consensus.cc:3037] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.627234 8152 raft_consensus.cc:504] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.627537 8152 leader_election.cc:288] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:20.627878 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2e6050128b45f3b6866a6770ebafb8" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:20.627996 7013 raft_consensus.cc:3037] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.628705 7013 raft_consensus.cc:2454] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:20.628960 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2e6050128b45f3b6866a6770ebafb8" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:20.628989 7077 leader_election.cc:302] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:20.629086 6880 raft_consensus.cc:3037] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.629151 8152 raft_consensus.cc:2781] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:20.629230 8152 raft_consensus.cc:686] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:20.629377 8152 consensus_queue.cc:237] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:20.630057 6880 raft_consensus.cc:2454] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:20.631153 6732 catalog_manager.cc:5484] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:20.735880 7923 raft_consensus.cc:482] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:20.736063 7923 raft_consensus.cc:504] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:20.736514 7923 leader_election.cc:288] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:20.736922 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb2d524281d43e4bbb962ed7b71e330" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:20.736923 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb2d524281d43e4bbb962ed7b71e330" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:20.737049 7013 raft_consensus.cc:2454] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:20.737066 6880 raft_consensus.cc:2454] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:20.737367 7208 leader_election.cc:302] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:20.737529 7923 raft_consensus.cc:2781] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:20.737586 7923 raft_consensus.cc:482] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:20.737612 7923 raft_consensus.cc:3037] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.738363 7923 raft_consensus.cc:504] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:20.738680 7923 leader_election.cc:288] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:20.739043 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb2d524281d43e4bbb962ed7b71e330" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:20.739050 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb2d524281d43e4bbb962ed7b71e330" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:20.739157 6880 raft_consensus.cc:3037] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.739172 7013 raft_consensus.cc:3037] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.739884 6880 raft_consensus.cc:2454] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:20.739924 7013 raft_consensus.cc:2454] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:20.740181 7210 leader_election.cc:302] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:20.740442 7923 raft_consensus.cc:2781] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:20.740526 7923 raft_consensus.cc:686] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:20.740661 7923 consensus_queue.cc:237] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:20.742388 6732 catalog_manager.cc:5484] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:20.776937 8152 raft_consensus.cc:482] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:20.777070 8152 raft_consensus.cc:504] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.777333 7923 raft_consensus.cc:482] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:20.777459 7923 raft_consensus.cc:504] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.778235 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015b439880d94914b0571789407be4f9" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:20.778337 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015b439880d94914b0571789407be4f9" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:20.778381 7144 raft_consensus.cc:2454] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:20.778473 6881 raft_consensus.cc:2454] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:20.778714 7208 leader_election.cc:302] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:20.778240 8152 leader_election.cc:288] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:20.778965 7970 raft_consensus.cc:2781] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:20.779042 7970 raft_consensus.cc:482] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:20.779091 7970 raft_consensus.cc:3037] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.779129 7923 leader_election.cc:288] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:20.779160 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015b439880d94914b0571789407be4f9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:20.779274 6880 raft_consensus.cc:2454] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:20.779551 7079 leader_election.cc:302] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:20.779727 8152 raft_consensus.cc:2781] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:20.779784 8152 raft_consensus.cc:482] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:20.779826 8152 raft_consensus.cc:3037] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.780138 7970 raft_consensus.cc:504] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.780488 7970 leader_election.cc:288] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:20.780757 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015b439880d94914b0571789407be4f9" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:20.780856 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015b439880d94914b0571789407be4f9" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:20.780947 6880 raft_consensus.cc:3037] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:20.781282 8152 raft_consensus.cc:504] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.781527 7144 raft_consensus.cc:2379] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e3394a4be9ea4def8aa39b333c82c5ff in current term 1: Already voted for candidate 1aaf8befba984e199e96f53f46b50441 in this term.
I20240426 09:49:20.781845 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015b439880d94914b0571789407be4f9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:20.781899 8152 leader_election.cc:288] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:20.782059 6880 raft_consensus.cc:2454] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:20.782271 7210 leader_election.cc:302] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters: 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:20.782451 7970 raft_consensus.cc:2781] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:20.782534 7970 raft_consensus.cc:686] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:20.782660 7970 consensus_queue.cc:237] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:20.783047 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015b439880d94914b0571789407be4f9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:20.783094 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015b439880d94914b0571789407be4f9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:20.783524 7079 leader_election.cc:302] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441; no voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:20.783706 8152 raft_consensus.cc:2726] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240426 09:49:20.784394 6732 catalog_manager.cc:5484] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
W20240426 09:49:20.942415 8188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:20.942685 8188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:20.945636 8188 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240426 09:49:21.021179 7013 raft_consensus.cc:1261] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.021596 7970 consensus_queue.cc:1035] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.022090 6881 raft_consensus.cc:1261] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.022090 6880 raft_consensus.cc:1261] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.023839 7144 raft_consensus.cc:1261] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.024674 7970 consensus_queue.cc:1035] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.024930 7970 consensus_queue.cc:1035] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.025138 7970 consensus_queue.cc:1035] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.035008 6882 raft_consensus.cc:1261] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.039144 8152 consensus_queue.cc:1035] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.042340 7013 raft_consensus.cc:1261] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.042735 8152 consensus_queue.cc:1035] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 93.7145 ms
time per row: 0.0468573 ms
I20240426 09:49:21.082221 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:21.087905 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 5ba744a156204d9e81db6ea113945203 (DEFAULT_TABLE table=rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.087873 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 5ba744a156204d9e81db6ea113945203 (DEFAULT_TABLE table=rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.088184 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba744a156204d9e81db6ea113945203. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.088184 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba744a156204d9e81db6ea113945203. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.088115 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 5ba744a156204d9e81db6ea113945203 (DEFAULT_TABLE table=rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.088411 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba744a156204d9e81db6ea113945203. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.090338 8248 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:21.090518 7254 tablet_service.cc:1467] Processing CreateTablet for tablet e5baaf8ce5a247e2975f8d6327349a26 (DEFAULT_TABLE table=rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.090667 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 956f89ba28c54473964cc877c26580f0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.090747 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5baaf8ce5a247e2975f8d6327349a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.090801 8249 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:21.090940 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 956f89ba28c54473964cc877c26580f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.091578 8249 tablet_bootstrap.cc:654] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.092880 8248 tablet_bootstrap.cc:654] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.093191 7124 tablet_service.cc:1467] Processing CreateTablet for tablet 956f89ba28c54473964cc877c26580f0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.093291 7122 tablet_service.cc:1467] Processing CreateTablet for tablet e5baaf8ce5a247e2975f8d6327349a26 (DEFAULT_TABLE table=rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.093413 7124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 956f89ba28c54473964cc877c26580f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.093591 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5baaf8ce5a247e2975f8d6327349a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.093611 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 956f89ba28c54473964cc877c26580f0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.093844 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 956f89ba28c54473964cc877c26580f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.093869 8249 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:21.093941 8249 ts_tablet_manager.cc:1367] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:49:21.094354 8249 raft_consensus.cc:348] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.094466 8249 raft_consensus.cc:374] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.094501 8249 raft_consensus.cc:729] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.094633 8249 consensus_queue.cc:260] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.095487 8249 ts_tablet_manager.cc:1398] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:21.095645 8252 tablet_bootstrap.cc:492] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:21.095646 8249 tablet_bootstrap.cc:492] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:21.096598 8249 tablet_bootstrap.cc:654] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.096627 8252 tablet_bootstrap.cc:654] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.097065 6991 tablet_service.cc:1467] Processing CreateTablet for tablet e5baaf8ce5a247e2975f8d6327349a26 (DEFAULT_TABLE table=rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.097276 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5baaf8ce5a247e2975f8d6327349a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.097962 8249 tablet_bootstrap.cc:492] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:21.098042 8249 ts_tablet_manager.cc:1367] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240426 09:49:21.098393 8249 raft_consensus.cc:348] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.098477 8249 raft_consensus.cc:374] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.098512 8249 raft_consensus.cc:729] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.098639 8249 consensus_queue.cc:260] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.098981 8249 ts_tablet_manager.cc:1398] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:21.099165 8249 tablet_bootstrap.cc:492] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:21.099694 8248 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:21.099741 8250 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:21.099789 8248 ts_tablet_manager.cc:1367] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240426 09:49:21.100265 8248 raft_consensus.cc:348] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.100399 8248 raft_consensus.cc:374] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.100457 8248 raft_consensus.cc:729] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.100617 8250 tablet_bootstrap.cc:654] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.100575 8248 consensus_queue.cc:260] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.100920 8248 ts_tablet_manager.cc:1398] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:21.101076 8248 tablet_bootstrap.cc:492] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:21.101784 8250 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:21.101897 8250 ts_tablet_manager.cc:1367] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:21.102016 8248 tablet_bootstrap.cc:654] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.102052 8252 tablet_bootstrap.cc:492] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:21.102126 8252 ts_tablet_manager.cc:1367] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:21.102332 8250 raft_consensus.cc:348] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.102437 8250 raft_consensus.cc:374] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.102468 8250 raft_consensus.cc:729] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.102586 8252 raft_consensus.cc:348] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:21.102686 8252 raft_consensus.cc:374] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.102643 8250 consensus_queue.cc:260] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.103032 8250 ts_tablet_manager.cc:1398] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:21.103194 8250 tablet_bootstrap.cc:492] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:21.103600 8249 tablet_bootstrap.cc:654] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.104537 8250 tablet_bootstrap.cc:654] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.102715 8252 raft_consensus.cc:729] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.105082 8248 tablet_bootstrap.cc:492] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:21.105166 8248 ts_tablet_manager.cc:1367] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:21.105122 8252 consensus_queue.cc:260] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:21.105463 8252 ts_tablet_manager.cc:1398] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:21.105549 8250 tablet_bootstrap.cc:492] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:21.105548 8248 raft_consensus.cc:348] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:21.105625 8250 ts_tablet_manager.cc:1367] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:21.105669 8248 raft_consensus.cc:374] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.105715 8248 raft_consensus.cc:729] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.105829 8248 consensus_queue.cc:260] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:21.106082 8248 ts_tablet_manager.cc:1398] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:21.106048 8250 raft_consensus.cc:348] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.106139 8250 raft_consensus.cc:374] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.106181 8250 raft_consensus.cc:729] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.106294 8248 tablet_bootstrap.cc:492] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:21.106297 8250 consensus_queue.cc:260] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.106564 8250 ts_tablet_manager.cc:1398] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:21.107224 8248 tablet_bootstrap.cc:654] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.108407 8249 tablet_bootstrap.cc:492] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:21.108505 8249 ts_tablet_manager.cc:1367] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:21.109023 8249 raft_consensus.cc:348] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:21.109138 8249 raft_consensus.cc:374] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.109171 8249 raft_consensus.cc:729] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.109295 8249 consensus_queue.cc:260] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:21.109625 8249 ts_tablet_manager.cc:1398] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:21.110309 8248 tablet_bootstrap.cc:492] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:21.110397 8248 ts_tablet_manager.cc:1367] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:21.110926 8248 raft_consensus.cc:348] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.111023 8248 raft_consensus.cc:374] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.111054 8248 raft_consensus.cc:729] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.111179 8248 consensus_queue.cc:260] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.111455 8248 ts_tablet_manager.cc:1398] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:21.154558 8152 raft_consensus.cc:482] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:21.154698 8152 raft_consensus.cc:504] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:21.155198 8152 leader_election.cc:288] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:21.155596 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "956f89ba28c54473964cc877c26580f0" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:21.155612 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "956f89ba28c54473964cc877c26580f0" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:21.155746 7013 raft_consensus.cc:2454] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:21.155750 6881 raft_consensus.cc:2454] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:21.156056 7079 leader_election.cc:302] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:21.156237 8152 raft_consensus.cc:2781] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:21.156292 8152 raft_consensus.cc:482] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:21.156337 8152 raft_consensus.cc:3037] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.157152 8152 raft_consensus.cc:504] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:21.157475 8152 leader_election.cc:288] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:21.157824 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "956f89ba28c54473964cc877c26580f0" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:21.157855 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "956f89ba28c54473964cc877c26580f0" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:21.157958 6881 raft_consensus.cc:3037] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.157959 7013 raft_consensus.cc:3037] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.159070 7013 raft_consensus.cc:2454] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:21.159379 6881 raft_consensus.cc:2454] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:21.159389 7077 leader_election.cc:302] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:21.159546 8152 raft_consensus.cc:2781] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:21.159621 8152 raft_consensus.cc:686] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:21.159754 8152 consensus_queue.cc:237] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:21.161567 6738 catalog_manager.cc:5484] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:21.362815 8165 raft_consensus.cc:482] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:21.362962 8165 raft_consensus.cc:504] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.363445 8165 leader_election.cc:288] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:21.363886 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5baaf8ce5a247e2975f8d6327349a26" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:21.363898 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5baaf8ce5a247e2975f8d6327349a26" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:21.364037 7275 raft_consensus.cc:2454] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:21.364039 7144 raft_consensus.cc:2454] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:21.364354 6948 leader_election.cc:302] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:21.364507 8165 raft_consensus.cc:2781] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:21.364565 8165 raft_consensus.cc:482] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:21.364611 8165 raft_consensus.cc:3037] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.365417 8165 raft_consensus.cc:504] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.365729 8165 leader_election.cc:288] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:21.366101 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5baaf8ce5a247e2975f8d6327349a26" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:21.366103 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5baaf8ce5a247e2975f8d6327349a26" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:21.366241 7144 raft_consensus.cc:3037] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.366241 7275 raft_consensus.cc:3037] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.367189 7144 raft_consensus.cc:2454] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:21.367189 7275 raft_consensus.cc:2454] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:21.367480 6948 leader_election.cc:302] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:21.367630 8165 raft_consensus.cc:2781] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:21.367867 8165 raft_consensus.cc:686] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:21.368026 8165 consensus_queue.cc:237] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.369678 6738 catalog_manager.cc:5484] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:21.370641 8106 raft_consensus.cc:482] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:21.370720 8106 raft_consensus.cc:504] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.371105 8106 leader_election.cc:288] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:21.371441 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba744a156204d9e81db6ea113945203" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:21.371464 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba744a156204d9e81db6ea113945203" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:21.371565 7013 raft_consensus.cc:2454] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:21.371579 7275 raft_consensus.cc:2454] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:21.371860 7079 leader_election.cc:302] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:21.372009 8106 raft_consensus.cc:2781] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:21.372067 8106 raft_consensus.cc:482] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:21.372093 8106 raft_consensus.cc:3037] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.372889 8106 raft_consensus.cc:504] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.373237 8106 leader_election.cc:288] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:21.373502 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba744a156204d9e81db6ea113945203" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:21.373536 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba744a156204d9e81db6ea113945203" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:21.373596 7013 raft_consensus.cc:3037] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.373629 7275 raft_consensus.cc:3037] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.374305 7275 raft_consensus.cc:2454] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:21.374553 7079 leader_election.cc:302] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:21.374709 8106 raft_consensus.cc:2781] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:21.374830 8106 raft_consensus.cc:686] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:21.374971 8106 consensus_queue.cc:237] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.376538 6738 catalog_manager.cc:5484] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:21.391798 7013 raft_consensus.cc:2454] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
W20240426 09:49:21.487334 8265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:21.487555 8265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:21.490639 8265 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240426 09:49:21.554886 8106 consensus_queue.cc:1035] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.558426 8152 consensus_queue.cc:1035] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.577188 7275 raft_consensus.cc:1261] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.577553 7144 raft_consensus.cc:1261] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.578050 8165 consensus_queue.cc:1035] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.578330 8165 consensus_queue.cc:1035] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.582222 7274 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.582614 8152 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.588130 7013 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:21.588747 8106 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:21.590868 8303 mvcc.cc:205] Tried to move back new op lower bound from 7021055842614116352 to 7021055841764573184. Current Snapshot: MvccSnapshot[applied={T|T < 7021055842614116352}]
I20240426 09:49:21.609141 8301 mvcc.cc:205] Tried to move back new op lower bound from 7021055842635415552 to 7021055841793208320. Current Snapshot: MvccSnapshot[applied={T|T < 7021055842635415552}]
I20240426 09:49:21.614941 8311 mvcc.cc:205] Tried to move back new op lower bound from 7021055842635415552 to 7021055841793208320. Current Snapshot: MvccSnapshot[applied={T|T < 7021055842635415552}]
INSERT report
rows total: 2000
time total: 103.749 ms
time per row: 0.0518745 ms
I20240426 09:49:21.637997 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:21.643530 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 74c34398941848f5989acb2a1a88a82f (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.643772 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 69cb7b27400b4adc85071876764609f9 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.643883 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74c34398941848f5989acb2a1a88a82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.644064 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69cb7b27400b4adc85071876764609f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.644373 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 74c34398941848f5989acb2a1a88a82f (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.644578 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74c34398941848f5989acb2a1a88a82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.644559 7124 tablet_service.cc:1467] Processing CreateTablet for tablet 69cb7b27400b4adc85071876764609f9 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.644819 7124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69cb7b27400b4adc85071876764609f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.645175 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 69cb7b27400b4adc85071876764609f9 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.645220 7255 tablet_service.cc:1467] Processing CreateTablet for tablet c4bb42dad9a04eb78e0ee6453c8693b7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.645378 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69cb7b27400b4adc85071876764609f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.645552 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4bb42dad9a04eb78e0ee6453c8693b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.646806 8324 tablet_bootstrap.cc:492] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:21.647210 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 74c34398941848f5989acb2a1a88a82f (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.647289 6993 tablet_service.cc:1467] Processing CreateTablet for tablet c4bb42dad9a04eb78e0ee6453c8693b7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.647352 6860 tablet_service.cc:1467] Processing CreateTablet for tablet c4bb42dad9a04eb78e0ee6453c8693b7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:21.647473 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74c34398941848f5989acb2a1a88a82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.647553 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4bb42dad9a04eb78e0ee6453c8693b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.647815 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4bb42dad9a04eb78e0ee6453c8693b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:21.648288 8324 tablet_bootstrap.cc:654] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.649340 8326 tablet_bootstrap.cc:492] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:21.649492 8324 tablet_bootstrap.cc:492] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:21.649577 8324 ts_tablet_manager.cc:1367] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:21.650188 8326 tablet_bootstrap.cc:654] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.650110 8324 raft_consensus.cc:348] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.650244 8324 raft_consensus.cc:374] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.650285 8324 raft_consensus.cc:729] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.650439 8324 consensus_queue.cc:260] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.650833 8324 ts_tablet_manager.cc:1398] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:21.651712 8326 tablet_bootstrap.cc:492] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:21.651798 8326 ts_tablet_manager.cc:1367] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:21.652173 8326 raft_consensus.cc:348] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.652294 8326 raft_consensus.cc:374] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.652338 8326 raft_consensus.cc:729] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.652459 8326 consensus_queue.cc:260] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.652782 8326 ts_tablet_manager.cc:1398] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:21.653053 8326 tablet_bootstrap.cc:492] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:21.654011 8326 tablet_bootstrap.cc:654] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.655309 8326 tablet_bootstrap.cc:492] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:21.655390 8326 ts_tablet_manager.cc:1367] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:21.655812 8326 raft_consensus.cc:348] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.655896 8326 raft_consensus.cc:374] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.655936 8326 raft_consensus.cc:729] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.656055 8326 consensus_queue.cc:260] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.656296 8326 ts_tablet_manager.cc:1398] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:21.665350 8329 tablet_bootstrap.cc:492] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:21.666422 8329 tablet_bootstrap.cc:654] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.666464 8330 tablet_bootstrap.cc:492] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:21.667256 8330 tablet_bootstrap.cc:654] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.667495 8329 tablet_bootstrap.cc:492] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:21.667560 8329 ts_tablet_manager.cc:1367] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:21.667903 8329 raft_consensus.cc:348] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.667999 8329 raft_consensus.cc:374] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.668020 8329 raft_consensus.cc:729] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.668140 8329 consensus_queue.cc:260] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.668187 8330 tablet_bootstrap.cc:492] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:21.668254 8330 ts_tablet_manager.cc:1367] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:21.668390 8329 ts_tablet_manager.cc:1398] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:21.668531 8329 tablet_bootstrap.cc:492] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:21.668658 8330 raft_consensus.cc:348] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.668759 8330 raft_consensus.cc:374] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.668788 8330 raft_consensus.cc:729] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.668893 8330 consensus_queue.cc:260] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.669155 8330 ts_tablet_manager.cc:1398] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:21.669334 8330 tablet_bootstrap.cc:492] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:21.669831 8329 tablet_bootstrap.cc:654] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.671034 8330 tablet_bootstrap.cc:654] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.671336 8329 tablet_bootstrap.cc:492] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:21.671415 8329 ts_tablet_manager.cc:1367] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:21.671826 8329 raft_consensus.cc:348] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.671928 8329 raft_consensus.cc:374] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.671965 8329 raft_consensus.cc:729] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.672096 8329 consensus_queue.cc:260] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.672219 8330 tablet_bootstrap.cc:492] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:21.672300 8330 ts_tablet_manager.cc:1367] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:21.672572 8329 ts_tablet_manager.cc:1398] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:21.672729 8329 tablet_bootstrap.cc:492] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:21.672703 8330 raft_consensus.cc:348] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.672803 8330 raft_consensus.cc:374] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.672839 8330 raft_consensus.cc:729] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.672977 8330 consensus_queue.cc:260] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.673259 8330 ts_tablet_manager.cc:1398] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:21.673523 8330 tablet_bootstrap.cc:492] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:21.673909 8329 tablet_bootstrap.cc:654] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.674448 8330 tablet_bootstrap.cc:654] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:21.675606 8330 tablet_bootstrap.cc:492] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:21.675704 8330 ts_tablet_manager.cc:1367] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:21.675879 8329 tablet_bootstrap.cc:492] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:21.675961 8329 ts_tablet_manager.cc:1367] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:21.676098 8330 raft_consensus.cc:348] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.676180 8330 raft_consensus.cc:374] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.676204 8330 raft_consensus.cc:729] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.676321 8330 consensus_queue.cc:260] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.676395 8329 raft_consensus.cc:348] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.676506 8329 raft_consensus.cc:374] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:21.676543 8329 raft_consensus.cc:729] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:21.676611 8330 ts_tablet_manager.cc:1398] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:21.677054 8329 consensus_queue.cc:260] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.677408 8329 ts_tablet_manager.cc:1398] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:21.844520 8165 raft_consensus.cc:482] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:21.844677 8165 raft_consensus.cc:504] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.845052 8165 leader_election.cc:288] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:21.845566 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c34398941848f5989acb2a1a88a82f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:21.845566 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c34398941848f5989acb2a1a88a82f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:21.845723 6881 raft_consensus.cc:2454] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:21.845724 7274 raft_consensus.cc:2454] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:21.846055 6948 leader_election.cc:302] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:21.846226 8165 raft_consensus.cc:2781] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:21.846283 8165 raft_consensus.cc:482] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:21.846333 8165 raft_consensus.cc:3037] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.847169 8165 raft_consensus.cc:504] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.847474 8165 leader_election.cc:288] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:21.847841 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c34398941848f5989acb2a1a88a82f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:21.847872 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c34398941848f5989acb2a1a88a82f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:21.847975 7274 raft_consensus.cc:3037] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.847975 6881 raft_consensus.cc:3037] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.848932 7274 raft_consensus.cc:2454] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:21.848930 6881 raft_consensus.cc:2454] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:21.849236 6948 leader_election.cc:302] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:21.849390 8165 raft_consensus.cc:2781] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:21.849471 8165 raft_consensus.cc:686] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:21.849620 8165 consensus_queue.cc:237] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:21.851400 6736 catalog_manager.cc:5484] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:21.923513 8165 raft_consensus.cc:482] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:21.923648 8165 raft_consensus.cc:504] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.924098 8165 leader_election.cc:288] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:21.924536 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4bb42dad9a04eb78e0ee6453c8693b7" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:21.924597 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4bb42dad9a04eb78e0ee6453c8693b7" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:21.924691 6881 raft_consensus.cc:2454] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:21.924724 7274 raft_consensus.cc:2454] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:21.925009 6948 leader_election.cc:302] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:21.925225 8165 raft_consensus.cc:2781] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:21.925289 8165 raft_consensus.cc:482] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:21.925343 8165 raft_consensus.cc:3037] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.926144 8165 raft_consensus.cc:504] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.926476 8165 leader_election.cc:288] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:21.926872 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4bb42dad9a04eb78e0ee6453c8693b7" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:21.926877 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4bb42dad9a04eb78e0ee6453c8693b7" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:21.926976 6881 raft_consensus.cc:3037] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.927004 7274 raft_consensus.cc:3037] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.927716 6881 raft_consensus.cc:2454] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:21.927935 7274 raft_consensus.cc:2454] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:21.927970 6948 leader_election.cc:302] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:21.928138 8165 raft_consensus.cc:2781] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:21.928222 8165 raft_consensus.cc:686] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:21.928365 8165 consensus_queue.cc:237] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.930064 6736 catalog_manager.cc:5484] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:21.970201 8152 raft_consensus.cc:482] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:21.970311 8152 raft_consensus.cc:504] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.970625 8152 leader_election.cc:288] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:21.970995 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69cb7b27400b4adc85071876764609f9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:21.971045 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69cb7b27400b4adc85071876764609f9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:21.971117 6881 raft_consensus.cc:2454] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:21.971190 7274 raft_consensus.cc:2454] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:21.971395 7079 leader_election.cc:302] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:21.971560 8152 raft_consensus.cc:2781] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:21.971619 8152 raft_consensus.cc:482] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:21.971668 8152 raft_consensus.cc:3037] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.972473 8152 raft_consensus.cc:504] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.972754 8152 leader_election.cc:288] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:21.973084 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69cb7b27400b4adc85071876764609f9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:21.973100 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69cb7b27400b4adc85071876764609f9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:21.973182 7274 raft_consensus.cc:3037] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.973183 6881 raft_consensus.cc:3037] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:21.973893 7274 raft_consensus.cc:2454] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:21.973995 6881 raft_consensus.cc:2454] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:21.974164 7079 leader_election.cc:302] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:21.974321 8152 raft_consensus.cc:2781] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:21.974411 8152 raft_consensus.cc:686] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:21.974565 8152 consensus_queue.cc:237] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:21.976306 6736 catalog_manager.cc:5484] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
W20240426 09:49:22.119103 8340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:22.119349 8340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:22.122243 8340 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240426 09:49:22.214249 7274 raft_consensus.cc:1261] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.214790 8152 consensus_queue.cc:1035] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.216894 6881 raft_consensus.cc:1261] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.217307 8152 consensus_queue.cc:1035] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.217511 7275 raft_consensus.cc:1261] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.217808 8165 consensus_queue.cc:1035] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.219110 6881 raft_consensus.cc:1261] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.219589 8165 consensus_queue.cc:1035] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.222199 6882 raft_consensus.cc:1261] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.222532 8165 consensus_queue.cc:1035] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.224223 7273 raft_consensus.cc:1261] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.224612 8165 consensus_queue.cc:1035] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.228124 8373 mvcc.cc:205] Tried to move back new op lower bound from 7021055845224697856 to 7021055844249059328. Current Snapshot: MvccSnapshot[applied={T|T < 7021055845224697856}]
INSERT report
rows total: 2000
time total: 115.439 ms
time per row: 0.0577193 ms
I20240426 09:49:22.283480 6735 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:22.288509 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa5a98645834e2886ff45b8b28c1c91 (DEFAULT_TABLE table=rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.288559 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa5a98645834e2886ff45b8b28c1c91 (DEFAULT_TABLE table=rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.288823 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa5a98645834e2886ff45b8b28c1c91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.288859 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa5a98645834e2886ff45b8b28c1c91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.289592 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 5951b29e98cc4ff3a459dec654580ec0 (DEFAULT_TABLE table=rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.289817 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5951b29e98cc4ff3a459dec654580ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.290055 7124 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa5a98645834e2886ff45b8b28c1c91 (DEFAULT_TABLE table=rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.290309 7124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa5a98645834e2886ff45b8b28c1c91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.291381 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 5951b29e98cc4ff3a459dec654580ec0 (DEFAULT_TABLE table=rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.291456 8400 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:22.291649 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5951b29e98cc4ff3a459dec654580ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.292133 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 5951b29e98cc4ff3a459dec654580ec0 (DEFAULT_TABLE table=rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.292330 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5951b29e98cc4ff3a459dec654580ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.292515 8400 tablet_bootstrap.cc:654] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.292635 7123 tablet_service.cc:1467] Processing CreateTablet for tablet f834d8a08b1341869fd644ff400fceae (DEFAULT_TABLE table=rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.292852 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f834d8a08b1341869fd644ff400fceae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.293838 8403 tablet_bootstrap.cc:492] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:22.294235 7253 tablet_service.cc:1467] Processing CreateTablet for tablet f834d8a08b1341869fd644ff400fceae (DEFAULT_TABLE table=rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.294453 8399 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:22.294479 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f834d8a08b1341869fd644ff400fceae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.294706 8403 tablet_bootstrap.cc:654] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.295292 8399 tablet_bootstrap.cc:654] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.295730 8401 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:22.296070 6993 tablet_service.cc:1467] Processing CreateTablet for tablet f834d8a08b1341869fd644ff400fceae (DEFAULT_TABLE table=rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.296295 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f834d8a08b1341869fd644ff400fceae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.297147 8399 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:22.297241 8401 tablet_bootstrap.cc:654] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.297396 8399 ts_tablet_manager.cc:1367] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:22.297881 8399 raft_consensus.cc:348] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.298002 8399 raft_consensus.cc:374] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.298043 8399 raft_consensus.cc:729] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.298174 8399 consensus_queue.cc:260] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.298476 8401 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:22.298550 8401 ts_tablet_manager.cc:1367] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:22.298555 8399 ts_tablet_manager.cc:1398] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.298784 8399 tablet_bootstrap.cc:492] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:22.299052 8400 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:22.299149 8400 ts_tablet_manager.cc:1367] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:22.299463 8401 raft_consensus.cc:348] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.299580 8401 raft_consensus.cc:374] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.299607 8399 tablet_bootstrap.cc:654] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.299615 8401 raft_consensus.cc:729] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.299631 8400 raft_consensus.cc:348] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.299739 8400 raft_consensus.cc:374] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.299773 8400 raft_consensus.cc:729] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.299768 8401 consensus_queue.cc:260] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.299909 8400 consensus_queue.cc:260] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.300153 8401 ts_tablet_manager.cc:1398] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:22.300207 8400 ts_tablet_manager.cc:1398] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.300318 8401 tablet_bootstrap.cc:492] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:22.300379 8400 tablet_bootstrap.cc:492] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:22.300735 8403 tablet_bootstrap.cc:492] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:22.300873 8403 ts_tablet_manager.cc:1367] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:22.301129 8401 tablet_bootstrap.cc:654] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.301266 8400 tablet_bootstrap.cc:654] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.302311 8400 tablet_bootstrap.cc:492] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:22.302388 8400 ts_tablet_manager.cc:1367] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:22.302824 8400 raft_consensus.cc:348] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.302920 8400 raft_consensus.cc:374] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.302963 8400 raft_consensus.cc:729] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.303068 8400 consensus_queue.cc:260] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.303350 8400 ts_tablet_manager.cc:1398] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.304386 8403 raft_consensus.cc:348] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.304502 8403 raft_consensus.cc:374] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.304638 8403 raft_consensus.cc:729] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.304778 8403 consensus_queue.cc:260] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.305058 8403 ts_tablet_manager.cc:1398] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.305220 8403 tablet_bootstrap.cc:492] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:22.306087 8403 tablet_bootstrap.cc:654] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.307286 8399 tablet_bootstrap.cc:492] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:22.307404 8399 ts_tablet_manager.cc:1367] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:22.307592 8401 tablet_bootstrap.cc:492] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:22.307701 8401 ts_tablet_manager.cc:1367] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:22.307866 8403 tablet_bootstrap.cc:492] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:22.307955 8403 ts_tablet_manager.cc:1367] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:22.307976 8399 raft_consensus.cc:348] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.308099 8399 raft_consensus.cc:374] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.308172 8399 raft_consensus.cc:729] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.308286 8401 raft_consensus.cc:348] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.308406 8401 raft_consensus.cc:374] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.308336 8399 consensus_queue.cc:260] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.308389 8403 raft_consensus.cc:348] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.308450 8401 raft_consensus.cc:729] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.308477 8403 raft_consensus.cc:374] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.308518 8403 raft_consensus.cc:729] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.308620 8403 consensus_queue.cc:260] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.308804 8399 ts_tablet_manager.cc:1398] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.308871 8403 ts_tablet_manager.cc:1398] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:22.308986 8401 consensus_queue.cc:260] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.309717 8401 ts_tablet_manager.cc:1398] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:22.309885 8401 tablet_bootstrap.cc:492] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:22.311960 8401 tablet_bootstrap.cc:654] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.313153 8401 tablet_bootstrap.cc:492] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:22.313231 8401 ts_tablet_manager.cc:1367] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:22.313531 8401 raft_consensus.cc:348] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.313607 8401 raft_consensus.cc:374] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.313635 8401 raft_consensus.cc:729] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.313735 8401 consensus_queue.cc:260] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.313974 8401 ts_tablet_manager.cc:1398] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:22.359983 8393 raft_consensus.cc:482] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:22.360098 8393 raft_consensus.cc:504] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.360489 8393 leader_election.cc:288] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:22.360838 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f834d8a08b1341869fd644ff400fceae" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:22.360873 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f834d8a08b1341869fd644ff400fceae" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:22.360966 7012 raft_consensus.cc:2454] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:22.361014 7274 raft_consensus.cc:2454] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:22.361258 7077 leader_election.cc:302] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:22.361485 8393 raft_consensus.cc:2781] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:22.361547 8393 raft_consensus.cc:482] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:22.361577 8393 raft_consensus.cc:3037] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.362406 8393 raft_consensus.cc:504] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.362715 8393 leader_election.cc:288] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:22.363080 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f834d8a08b1341869fd644ff400fceae" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:22.363206 7274 raft_consensus.cc:3037] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.363894 7274 raft_consensus.cc:2454] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:22.364121 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f834d8a08b1341869fd644ff400fceae" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:22.364171 7079 leader_election.cc:302] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:22.364221 7012 raft_consensus.cc:3037] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.364326 8393 raft_consensus.cc:2781] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:22.364403 8393 raft_consensus.cc:686] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:22.364545 8393 consensus_queue.cc:237] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.365059 7012 raft_consensus.cc:2454] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:22.366389 6738 catalog_manager.cc:5484] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:22.381556 7884 raft_consensus.cc:482] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:22.381701 7884 raft_consensus.cc:504] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.382117 7884 leader_election.cc:288] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:22.382524 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa5a98645834e2886ff45b8b28c1c91" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:22.382524 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa5a98645834e2886ff45b8b28c1c91" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:22.382665 7012 raft_consensus.cc:2454] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:22.382671 7144 raft_consensus.cc:2454] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:22.382977 6815 leader_election.cc:302] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:22.383133 7884 raft_consensus.cc:2781] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:22.383188 7884 raft_consensus.cc:482] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:22.383239 7884 raft_consensus.cc:3037] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.384057 7884 raft_consensus.cc:504] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.384372 7884 leader_election.cc:288] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:22.384693 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa5a98645834e2886ff45b8b28c1c91" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:22.384730 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa5a98645834e2886ff45b8b28c1c91" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:22.384830 7144 raft_consensus.cc:3037] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.384830 7012 raft_consensus.cc:3037] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.385726 7144 raft_consensus.cc:2454] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:22.385725 7012 raft_consensus.cc:2454] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:22.385988 6815 leader_election.cc:302] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:22.386127 7884 raft_consensus.cc:2781] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:22.386315 7884 raft_consensus.cc:686] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:22.386468 7884 consensus_queue.cc:237] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:22.388280 6738 catalog_manager.cc:5484] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:22.430912 7884 raft_consensus.cc:482] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:22.431030 7884 raft_consensus.cc:504] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.431396 7884 leader_election.cc:288] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:22.431793 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5951b29e98cc4ff3a459dec654580ec0" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:22.431793 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5951b29e98cc4ff3a459dec654580ec0" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:22.431938 7144 raft_consensus.cc:2454] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:22.431946 7274 raft_consensus.cc:2454] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:22.432214 6815 leader_election.cc:302] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:22.432374 7884 raft_consensus.cc:2781] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:22.432430 7884 raft_consensus.cc:482] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:22.432479 7884 raft_consensus.cc:3037] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.433355 7884 raft_consensus.cc:504] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.433652 7884 leader_election.cc:288] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:22.433969 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5951b29e98cc4ff3a459dec654580ec0" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:22.433969 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5951b29e98cc4ff3a459dec654580ec0" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:22.434087 7274 raft_consensus.cc:3037] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.434087 7144 raft_consensus.cc:3037] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.435021 7274 raft_consensus.cc:2454] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:22.435022 7144 raft_consensus.cc:2454] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:22.435323 6817 leader_election.cc:302] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:22.435494 7884 raft_consensus.cc:2781] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:22.435580 7884 raft_consensus.cc:686] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:22.435745 7884 consensus_queue.cc:237] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.437453 6738 catalog_manager.cc:5484] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
W20240426 09:49:22.561580 8416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:22.561820 8416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:22.564774 8416 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240426 09:49:22.638458 7144 raft_consensus.cc:1261] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.638998 7274 raft_consensus.cc:1261] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.639262 7275 raft_consensus.cc:1261] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.639657 7012 raft_consensus.cc:1261] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.640463 8152 consensus_queue.cc:1035] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.640792 8152 consensus_queue.cc:1035] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.641086 8445 consensus_queue.cc:1035] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.641475 8445 consensus_queue.cc:1035] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.642489 7013 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.645439 7144 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:22.646708 8413 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:22.647101 8413 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 106.205 ms
time per row: 0.0531024 ms
I20240426 09:49:22.710212 6736 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:22.715682 7123 tablet_service.cc:1467] Processing CreateTablet for tablet d47b2dc234b342119e5dafea80caecbb (DEFAULT_TABLE table=rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.715922 6861 tablet_service.cc:1467] Processing CreateTablet for tablet d47b2dc234b342119e5dafea80caecbb (DEFAULT_TABLE table=rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.716017 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47b2dc234b342119e5dafea80caecbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.716162 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47b2dc234b342119e5dafea80caecbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.716796 7253 tablet_service.cc:1467] Processing CreateTablet for tablet bcbf45071dd240c2ab6e75a9beac5b31 (DEFAULT_TABLE table=rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.717047 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbf45071dd240c2ab6e75a9beac5b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.717824 8401 tablet_bootstrap.cc:492] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:22.718041 6993 tablet_service.cc:1467] Processing CreateTablet for tablet d47b2dc234b342119e5dafea80caecbb (DEFAULT_TABLE table=rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.718283 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47b2dc234b342119e5dafea80caecbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.718243 6860 tablet_service.cc:1467] Processing CreateTablet for tablet bcbf45071dd240c2ab6e75a9beac5b31 (DEFAULT_TABLE table=rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.718433 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbf45071dd240c2ab6e75a9beac5b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.718644 8401 tablet_bootstrap.cc:654] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.719132 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 011cfc9512c54fe689ca5ba948e57fd8 (DEFAULT_TABLE table=rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.719375 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 011cfc9512c54fe689ca5ba948e57fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.720228 8399 tablet_bootstrap.cc:492] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:22.720371 8400 tablet_bootstrap.cc:492] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:22.720413 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 011cfc9512c54fe689ca5ba948e57fd8 (DEFAULT_TABLE table=rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.720628 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 011cfc9512c54fe689ca5ba948e57fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.721087 8399 tablet_bootstrap.cc:654] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.721385 8400 tablet_bootstrap.cc:654] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.722213 8399 tablet_bootstrap.cc:492] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:22.722307 8399 ts_tablet_manager.cc:1367] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:22.722575 8403 tablet_bootstrap.cc:492] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:22.722599 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 011cfc9512c54fe689ca5ba948e57fd8 (DEFAULT_TABLE table=rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.722719 8399 raft_consensus.cc:348] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.722860 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 011cfc9512c54fe689ca5ba948e57fd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.722868 8399 raft_consensus.cc:374] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.722903 8399 raft_consensus.cc:729] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.723038 8399 consensus_queue.cc:260] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.723305 8399 ts_tablet_manager.cc:1398] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.723485 8403 tablet_bootstrap.cc:654] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.724996 8400 tablet_bootstrap.cc:492] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:22.725078 8400 ts_tablet_manager.cc:1367] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:22.725132 8403 tablet_bootstrap.cc:492] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:22.725216 8403 ts_tablet_manager.cc:1367] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:22.725534 8400 raft_consensus.cc:348] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.725672 8400 raft_consensus.cc:374] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.725715 8400 raft_consensus.cc:729] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.725682 8403 raft_consensus.cc:348] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.725809 8403 raft_consensus.cc:374] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.725845 8403 raft_consensus.cc:729] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.725840 8400 consensus_queue.cc:260] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.725988 8403 consensus_queue.cc:260] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.726137 8400 ts_tablet_manager.cc:1398] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.726233 8403 ts_tablet_manager.cc:1398] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.726392 8403 tablet_bootstrap.cc:492] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:22.726387 8400 tablet_bootstrap.cc:492] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:22.727475 8400 tablet_bootstrap.cc:654] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.727479 6993 tablet_service.cc:1467] Processing CreateTablet for tablet bcbf45071dd240c2ab6e75a9beac5b31 (DEFAULT_TABLE table=rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:22.727734 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbf45071dd240c2ab6e75a9beac5b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:22.728649 8400 tablet_bootstrap.cc:492] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:22.728750 8400 ts_tablet_manager.cc:1367] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:22.729194 8400 raft_consensus.cc:348] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.729315 8400 raft_consensus.cc:374] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.729347 8400 raft_consensus.cc:729] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.729486 8400 consensus_queue.cc:260] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.729771 8399 tablet_bootstrap.cc:492] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:22.729871 8400 ts_tablet_manager.cc:1398] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:22.730051 8400 tablet_bootstrap.cc:492] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:22.730527 8399 tablet_bootstrap.cc:654] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.731079 8400 tablet_bootstrap.cc:654] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.731400 8403 tablet_bootstrap.cc:654] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.731699 8399 tablet_bootstrap.cc:492] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:22.731781 8399 ts_tablet_manager.cc:1367] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:22.732215 8399 raft_consensus.cc:348] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.732342 8399 raft_consensus.cc:374] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.732378 8399 raft_consensus.cc:729] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.732512 8399 consensus_queue.cc:260] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.732836 8399 ts_tablet_manager.cc:1398] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:22.733371 8403 tablet_bootstrap.cc:492] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:22.733461 8403 ts_tablet_manager.cc:1367] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:22.733884 8400 tablet_bootstrap.cc:492] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:22.733979 8400 ts_tablet_manager.cc:1367] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240426 09:49:22.734050 8403 raft_consensus.cc:348] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.734174 8403 raft_consensus.cc:374] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.734225 8403 raft_consensus.cc:729] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.734359 8403 consensus_queue.cc:260] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.734719 8403 ts_tablet_manager.cc:1398] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.734424 8400 raft_consensus.cc:348] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.735041 8400 raft_consensus.cc:374] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.735081 8400 raft_consensus.cc:729] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.735249 8400 consensus_queue.cc:260] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.735428 8401 tablet_bootstrap.cc:492] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:22.735507 8401 ts_tablet_manager.cc:1367] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240426 09:49:22.735584 8400 ts_tablet_manager.cc:1398] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:22.735924 8401 raft_consensus.cc:348] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.736037 8401 raft_consensus.cc:374] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.736073 8401 raft_consensus.cc:729] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.736183 8401 consensus_queue.cc:260] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.736431 8401 ts_tablet_manager.cc:1398] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.736604 8401 tablet_bootstrap.cc:492] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:22.737416 8401 tablet_bootstrap.cc:654] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:22.738457 8401 tablet_bootstrap.cc:492] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:22.738531 8401 ts_tablet_manager.cc:1367] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:22.738960 8401 raft_consensus.cc:348] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.739037 8401 raft_consensus.cc:374] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:22.739058 8401 raft_consensus.cc:729] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:22.739174 8401 consensus_queue.cc:260] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.739457 8401 ts_tablet_manager.cc:1398] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:22.776773 8444 raft_consensus.cc:482] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:22.776908 8444 raft_consensus.cc:504] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.777350 8444 leader_election.cc:288] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:22.777755 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011cfc9512c54fe689ca5ba948e57fd8" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:22.777783 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011cfc9512c54fe689ca5ba948e57fd8" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:22.777894 7143 raft_consensus.cc:2454] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:22.777935 7274 raft_consensus.cc:2454] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:22.778158 6815 leader_election.cc:302] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:22.778309 8444 raft_consensus.cc:2781] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:22.778363 8444 raft_consensus.cc:482] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:22.778395 8444 raft_consensus.cc:3037] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.779219 8444 raft_consensus.cc:504] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.779525 8444 leader_election.cc:288] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:22.779855 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011cfc9512c54fe689ca5ba948e57fd8" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:22.779846 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "011cfc9512c54fe689ca5ba948e57fd8" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:22.779965 7274 raft_consensus.cc:3037] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.779978 7143 raft_consensus.cc:3037] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.780699 7274 raft_consensus.cc:2454] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:22.780736 7143 raft_consensus.cc:2454] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:22.780987 6817 leader_election.cc:302] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:22.781159 8444 raft_consensus.cc:2781] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:22.781241 8444 raft_consensus.cc:686] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:22.781396 8444 consensus_queue.cc:237] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:22.783118 6737 catalog_manager.cc:5484] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:22.851119 8152 raft_consensus.cc:482] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:22.851276 8152 raft_consensus.cc:504] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.851707 8152 leader_election.cc:288] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:22.852147 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47b2dc234b342119e5dafea80caecbb" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:22.852161 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47b2dc234b342119e5dafea80caecbb" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:22.852288 6882 raft_consensus.cc:2454] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:22.852303 7012 raft_consensus.cc:2454] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:22.852604 7077 leader_election.cc:302] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:22.852756 8152 raft_consensus.cc:2781] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:22.852816 8152 raft_consensus.cc:482] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:22.852842 8152 raft_consensus.cc:3037] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.853758 8152 raft_consensus.cc:504] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.854091 8152 leader_election.cc:288] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:22.854434 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47b2dc234b342119e5dafea80caecbb" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:22.854465 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47b2dc234b342119e5dafea80caecbb" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:22.854543 7012 raft_consensus.cc:3037] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.854559 6882 raft_consensus.cc:3037] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:22.855376 6882 raft_consensus.cc:2454] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:22.855435 7012 raft_consensus.cc:2454] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:22.855672 7079 leader_election.cc:302] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:22.855908 8152 raft_consensus.cc:2781] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:22.856002 8152 raft_consensus.cc:686] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:22.856141 8152 consensus_queue.cc:237] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:22.857816 6737 catalog_manager.cc:5484] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:22.999506 8262 raft_consensus.cc:482] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:22.999647 8262 raft_consensus.cc:504] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.000080 8262 leader_election.cc:288] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:23.000597 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbf45071dd240c2ab6e75a9beac5b31" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:23.000663 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbf45071dd240c2ab6e75a9beac5b31" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:23.000723 6880 raft_consensus.cc:2454] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:23.000828 7274 raft_consensus.cc:2454] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:23.001044 6948 leader_election.cc:302] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:23.001252 8262 raft_consensus.cc:2781] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:23.001315 8262 raft_consensus.cc:482] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:23.001343 8262 raft_consensus.cc:3037] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.002178 8262 raft_consensus.cc:504] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.002482 8262 leader_election.cc:288] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:23.002856 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbf45071dd240c2ab6e75a9beac5b31" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:23.002923 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbf45071dd240c2ab6e75a9beac5b31" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:23.002986 6880 raft_consensus.cc:3037] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.003031 7274 raft_consensus.cc:3037] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.003744 7274 raft_consensus.cc:2454] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:23.003751 6880 raft_consensus.cc:2454] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:23.004035 6948 leader_election.cc:302] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:23.004189 8262 raft_consensus.cc:2781] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:23.004274 8262 raft_consensus.cc:686] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:23.004413 8262 consensus_queue.cc:237] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.006150 6737 catalog_manager.cc:5484] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
W20240426 09:49:23.125123 8466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:23.125360 8466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:23.128302 8466 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240426 09:49:23.212743 7143 raft_consensus.cc:1261] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.212745 7274 raft_consensus.cc:1261] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.213184 7884 consensus_queue.cc:1035] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.213344 8444 consensus_queue.cc:1035] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.213889 7274 raft_consensus.cc:1261] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.214030 6880 raft_consensus.cc:1261] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.214399 8165 consensus_queue.cc:1035] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.214594 8262 consensus_queue.cc:1035] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.221019 6880 raft_consensus.cc:1261] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.221482 8152 consensus_queue.cc:1035] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.224607 7012 raft_consensus.cc:1261] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.230926 8152 consensus_queue.cc:1035] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 115.696 ms
time per row: 0.0578479 ms
I20240426 09:49:23.286823 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:23.292342 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 0362b7d68d9048bca47d96e948563d03 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.292380 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 0362b7d68d9048bca47d96e948563d03 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.292654 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0362b7d68d9048bca47d96e948563d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.292654 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0362b7d68d9048bca47d96e948563d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.292765 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 325e1d125617456f877204314d9e7bb9 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.292941 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 0362b7d68d9048bca47d96e948563d03 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.293061 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325e1d125617456f877204314d9e7bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.293143 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0362b7d68d9048bca47d96e948563d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.295562 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 325e1d125617456f877204314d9e7bb9 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.295739 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 63f066721f3441299fe5dffd07c1afed (DEFAULT_TABLE table=rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.295810 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325e1d125617456f877204314d9e7bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.295853 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 63f066721f3441299fe5dffd07c1afed (DEFAULT_TABLE table=rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.295928 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f066721f3441299fe5dffd07c1afed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.296020 8529 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:23.296157 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f066721f3441299fe5dffd07c1afed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.296976 8529 tablet_bootstrap.cc:654] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.297835 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 63f066721f3441299fe5dffd07c1afed (DEFAULT_TABLE table=rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.298105 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f066721f3441299fe5dffd07c1afed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.298617 8531 tablet_bootstrap.cc:492] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:23.299454 8529 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:23.299489 8531 tablet_bootstrap.cc:654] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.299595 8529 ts_tablet_manager.cc:1367] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:23.300104 8529 raft_consensus.cc:348] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.300231 8529 raft_consensus.cc:374] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.300309 8529 raft_consensus.cc:729] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.300266 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 325e1d125617456f877204314d9e7bb9 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.300521 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325e1d125617456f877204314d9e7bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.300477 8529 consensus_queue.cc:260] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.300900 8529 ts_tablet_manager.cc:1398] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:23.301642 8529 tablet_bootstrap.cc:492] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:23.302419 8529 tablet_bootstrap.cc:654] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.302981 8531 tablet_bootstrap.cc:492] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:23.303084 8531 ts_tablet_manager.cc:1367] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:23.303580 8531 raft_consensus.cc:348] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.303725 8531 raft_consensus.cc:374] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.303735 8529 tablet_bootstrap.cc:492] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:23.303771 8531 raft_consensus.cc:729] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.303813 8529 ts_tablet_manager.cc:1367] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:23.303906 8531 consensus_queue.cc:260] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.304211 8531 ts_tablet_manager.cc:1398] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:23.304258 8529 raft_consensus.cc:348] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.304350 8529 raft_consensus.cc:374] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.304386 8529 raft_consensus.cc:729] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.304515 8529 consensus_queue.cc:260] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.304936 8528 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:23.305270 8529 ts_tablet_manager.cc:1398] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:23.306218 8528 tablet_bootstrap.cc:654] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.306995 8527 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:23.307503 8528 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:23.307590 8528 ts_tablet_manager.cc:1367] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240426 09:49:23.308043 8528 raft_consensus.cc:348] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.308133 8528 raft_consensus.cc:374] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.308168 8528 raft_consensus.cc:729] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.308274 8528 consensus_queue.cc:260] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.308537 8528 ts_tablet_manager.cc:1398] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:23.308697 8528 tablet_bootstrap.cc:492] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:23.309631 8528 tablet_bootstrap.cc:654] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.310739 8528 tablet_bootstrap.cc:492] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:23.310849 8528 ts_tablet_manager.cc:1367] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:23.311061 8527 tablet_bootstrap.cc:654] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.311347 8528 raft_consensus.cc:348] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.311451 8528 raft_consensus.cc:374] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.311484 8528 raft_consensus.cc:729] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.311632 8528 consensus_queue.cc:260] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.311930 8528 ts_tablet_manager.cc:1398] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:23.312103 8528 tablet_bootstrap.cc:492] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:23.312105 8527 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:23.312191 8527 ts_tablet_manager.cc:1367] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:23.312623 8527 raft_consensus.cc:348] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.312798 8527 raft_consensus.cc:374] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.312850 8527 raft_consensus.cc:729] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.313000 8528 tablet_bootstrap.cc:654] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.313081 8527 consensus_queue.cc:260] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.313905 8527 ts_tablet_manager.cc:1398] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:23.314066 8527 tablet_bootstrap.cc:492] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:23.314828 8527 tablet_bootstrap.cc:654] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.315918 8527 tablet_bootstrap.cc:492] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:23.315918 8528 tablet_bootstrap.cc:492] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:23.316004 8527 ts_tablet_manager.cc:1367] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:23.316004 8528 ts_tablet_manager.cc:1367] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240426 09:49:23.316429 8527 raft_consensus.cc:348] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.316429 8528 raft_consensus.cc:348] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.316520 8528 raft_consensus.cc:374] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.316520 8527 raft_consensus.cc:374] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.316552 8527 raft_consensus.cc:729] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.316552 8528 raft_consensus.cc:729] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.316682 8527 consensus_queue.cc:260] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.316673 8528 consensus_queue.cc:260] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.316934 8527 ts_tablet_manager.cc:1398] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:23.316944 8528 ts_tablet_manager.cc:1398] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:23.317137 8527 tablet_bootstrap.cc:492] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:23.317885 8527 tablet_bootstrap.cc:654] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.319777 8527 tablet_bootstrap.cc:492] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:23.319869 8527 ts_tablet_manager.cc:1367] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:23.320245 8527 raft_consensus.cc:348] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.320353 8527 raft_consensus.cc:374] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.320397 8527 raft_consensus.cc:729] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.320552 8527 consensus_queue.cc:260] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.320897 8527 ts_tablet_manager.cc:1398] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:23.391448 8152 raft_consensus.cc:482] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:23.391579 8152 raft_consensus.cc:504] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.392007 8152 leader_election.cc:288] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:23.392362 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f066721f3441299fe5dffd07c1afed" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:23.392356 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f066721f3441299fe5dffd07c1afed" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:23.392515 7012 raft_consensus.cc:2454] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:23.392524 6880 raft_consensus.cc:2454] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:23.392822 7079 leader_election.cc:302] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:23.393002 8152 raft_consensus.cc:2781] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:23.393064 8152 raft_consensus.cc:482] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:23.393090 8152 raft_consensus.cc:3037] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.393869 8152 raft_consensus.cc:504] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.394191 8152 leader_election.cc:288] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:23.394483 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f066721f3441299fe5dffd07c1afed" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:23.394484 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f066721f3441299fe5dffd07c1afed" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:23.394579 7012 raft_consensus.cc:3037] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.394591 6880 raft_consensus.cc:3037] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.395346 7012 raft_consensus.cc:2454] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:23.395370 6880 raft_consensus.cc:2454] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:23.395608 7077 leader_election.cc:302] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:23.395841 8152 raft_consensus.cc:2781] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:23.395911 8152 raft_consensus.cc:686] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:23.396047 8152 consensus_queue.cc:237] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.397688 6735 catalog_manager.cc:5484] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:23.416013 8152 raft_consensus.cc:482] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:23.416152 8152 raft_consensus.cc:504] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.416532 8152 leader_election.cc:288] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:23.416867 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0362b7d68d9048bca47d96e948563d03" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:23.416884 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0362b7d68d9048bca47d96e948563d03" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:23.416992 7012 raft_consensus.cc:2454] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:23.417028 7274 raft_consensus.cc:2454] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:23.417311 7077 leader_election.cc:302] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:23.417527 8152 raft_consensus.cc:2781] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:23.417591 8152 raft_consensus.cc:482] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:23.417629 8152 raft_consensus.cc:3037] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.418531 8152 raft_consensus.cc:504] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.418864 8152 leader_election.cc:288] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:23.419128 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0362b7d68d9048bca47d96e948563d03" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:23.419166 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0362b7d68d9048bca47d96e948563d03" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:23.419242 7012 raft_consensus.cc:3037] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.419266 7274 raft_consensus.cc:3037] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.419960 7274 raft_consensus.cc:2454] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:23.419963 7012 raft_consensus.cc:2454] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:23.420228 7079 leader_election.cc:302] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:23.420416 8152 raft_consensus.cc:2781] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:23.420496 8152 raft_consensus.cc:686] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:23.420632 8152 consensus_queue.cc:237] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:23.422478 6735 catalog_manager.cc:5484] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:23.489585 7970 raft_consensus.cc:482] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:23.489722 7970 raft_consensus.cc:504] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.490095 7970 leader_election.cc:288] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:23.490449 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325e1d125617456f877204314d9e7bb9" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:23.490538 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325e1d125617456f877204314d9e7bb9" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:23.490604 7012 raft_consensus.cc:2454] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:23.490693 7143 raft_consensus.cc:2454] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:23.490911 7208 leader_election.cc:302] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:23.491063 7970 raft_consensus.cc:2781] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:23.491119 7970 raft_consensus.cc:482] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:23.491144 7970 raft_consensus.cc:3037] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.491977 7970 raft_consensus.cc:504] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.492283 7970 leader_election.cc:288] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:23.492617 7012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325e1d125617456f877204314d9e7bb9" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:23.492638 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "325e1d125617456f877204314d9e7bb9" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:23.492748 7012 raft_consensus.cc:3037] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.492769 7143 raft_consensus.cc:3037] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.493724 7143 raft_consensus.cc:2454] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:23.493719 7012 raft_consensus.cc:2454] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:23.493996 7208 leader_election.cc:302] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:23.494141 7970 raft_consensus.cc:2781] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:23.494314 7970 raft_consensus.cc:686] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:23.494470 7970 consensus_queue.cc:237] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.496173 6735 catalog_manager.cc:5484] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
W20240426 09:49:23.639161 8544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:23.639431 8544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:23.642832 8544 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240426 09:49:23.723330 7012 raft_consensus.cc:1261] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.724025 6880 raft_consensus.cc:1261] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.724767 8393 consensus_queue.cc:1035] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.725208 8393 consensus_queue.cc:1035] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.725585 7013 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.726016 7274 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.728327 8152 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.731153 8393 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.733228 7011 raft_consensus.cc:1261] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.733891 7143 raft_consensus.cc:1261] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:23.734534 7970 consensus_queue.cc:1035] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.736542 7970 consensus_queue.cc:1035] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:23.763296 8498 mvcc.cc:205] Tried to move back new op lower bound from 7021055851406819328 to 7021055850071461888. Current Snapshot: MvccSnapshot[applied={T|T < 7021055851406819328 or (T in {7021055851406819328})}]
INSERT report
rows total: 2000
time total: 111.121 ms
time per row: 0.0555606 ms
I20240426 09:49:23.795478 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:23.800948 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 274ef2053af94bf1a038ec146c047063 (DEFAULT_TABLE table=rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.801098 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 274ef2053af94bf1a038ec146c047063 (DEFAULT_TABLE table=rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.801291 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 274ef2053af94bf1a038ec146c047063. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.801337 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 274ef2053af94bf1a038ec146c047063. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.803404 8529 tablet_bootstrap.cc:492] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:23.803751 8527 tablet_bootstrap.cc:492] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:23.803851 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 3f47b3eff1504b4d838c2afc3c0bb287 (DEFAULT_TABLE table=rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.803889 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 69179df98341493987f1d70ab24f62ef (DEFAULT_TABLE table=rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.804114 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69179df98341493987f1d70ab24f62ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.804118 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f47b3eff1504b4d838c2afc3c0bb287. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.804348 8529 tablet_bootstrap.cc:654] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.804785 8527 tablet_bootstrap.cc:654] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.805552 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 274ef2053af94bf1a038ec146c047063 (DEFAULT_TABLE table=rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.805775 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 274ef2053af94bf1a038ec146c047063. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.805873 8527 tablet_bootstrap.cc:492] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:23.805960 8527 ts_tablet_manager.cc:1367] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:23.806259 8531 tablet_bootstrap.cc:492] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:23.806407 8527 raft_consensus.cc:348] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:23.806516 8527 raft_consensus.cc:374] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.806562 8527 raft_consensus.cc:729] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.806648 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 69179df98341493987f1d70ab24f62ef (DEFAULT_TABLE table=rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.806677 8527 consensus_queue.cc:260] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:23.806960 8527 ts_tablet_manager.cc:1398] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:23.807103 8531 tablet_bootstrap.cc:654] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.807482 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69179df98341493987f1d70ab24f62ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.808176 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 3f47b3eff1504b4d838c2afc3c0bb287 (DEFAULT_TABLE table=rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.808351 8531 tablet_bootstrap.cc:492] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:23.808425 8531 ts_tablet_manager.cc:1367] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:23.808445 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f47b3eff1504b4d838c2afc3c0bb287. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.808823 8531 raft_consensus.cc:348] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.808907 8531 raft_consensus.cc:374] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.808931 8531 raft_consensus.cc:729] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.809028 8531 consensus_queue.cc:260] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.809275 8531 ts_tablet_manager.cc:1398] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:23.809444 8531 tablet_bootstrap.cc:492] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:23.809630 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 69179df98341493987f1d70ab24f62ef (DEFAULT_TABLE table=rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.809841 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69179df98341493987f1d70ab24f62ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.810158 8531 tablet_bootstrap.cc:654] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.810568 8528 tablet_bootstrap.cc:492] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:23.811373 8528 tablet_bootstrap.cc:654] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.812523 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 3f47b3eff1504b4d838c2afc3c0bb287 (DEFAULT_TABLE table=rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:23.813051 8531 tablet_bootstrap.cc:492] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:23.813131 8531 ts_tablet_manager.cc:1367] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:23.813179 8527 tablet_bootstrap.cc:492] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:23.813478 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f47b3eff1504b4d838c2afc3c0bb287. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:23.813560 8531 raft_consensus.cc:348] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:23.813658 8531 raft_consensus.cc:374] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.813694 8531 raft_consensus.cc:729] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.813813 8531 consensus_queue.cc:260] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:23.814383 8531 ts_tablet_manager.cc:1398] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:23.814533 8531 tablet_bootstrap.cc:492] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:23.817744 8527 tablet_bootstrap.cc:654] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.817874 8529 tablet_bootstrap.cc:492] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:23.817963 8529 ts_tablet_manager.cc:1367] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240426 09:49:23.817992 8531 tablet_bootstrap.cc:654] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.818280 8528 tablet_bootstrap.cc:492] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:23.818370 8528 ts_tablet_manager.cc:1367] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:23.818439 8529 raft_consensus.cc:348] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:23.818536 8529 raft_consensus.cc:374] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.818572 8529 raft_consensus.cc:729] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.818694 8529 consensus_queue.cc:260] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:23.818810 8527 tablet_bootstrap.cc:492] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:23.818789 8528 raft_consensus.cc:348] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.818877 8527 ts_tablet_manager.cc:1367] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:23.818897 8528 raft_consensus.cc:374] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.818928 8528 raft_consensus.cc:729] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.818998 8529 ts_tablet_manager.cc:1398] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:23.819038 8528 consensus_queue.cc:260] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.819149 8529 tablet_bootstrap.cc:492] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:23.819286 8528 ts_tablet_manager.cc:1398] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:23.819288 8527 raft_consensus.cc:348] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:23.819382 8527 raft_consensus.cc:374] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.819430 8527 raft_consensus.cc:729] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.819547 8527 consensus_queue.cc:260] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:23.819859 8527 ts_tablet_manager.cc:1398] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:23.820092 8529 tablet_bootstrap.cc:654] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.820492 8531 tablet_bootstrap.cc:492] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:23.820559 8531 ts_tablet_manager.cc:1367] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:23.821023 8531 raft_consensus.cc:348] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:23.821178 8529 tablet_bootstrap.cc:492] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:23.821254 8529 ts_tablet_manager.cc:1367] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:23.821619 8529 raft_consensus.cc:348] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:23.821733 8529 raft_consensus.cc:374] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.821774 8529 raft_consensus.cc:729] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.821874 8529 consensus_queue.cc:260] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:23.822170 8529 ts_tablet_manager.cc:1398] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:23.822301 8529 tablet_bootstrap.cc:492] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:23.823356 8529 tablet_bootstrap.cc:654] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:23.824254 8529 tablet_bootstrap.cc:492] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:23.824326 8529 ts_tablet_manager.cc:1367] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:23.824505 8531 raft_consensus.cc:374] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.824760 8529 raft_consensus.cc:348] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.824558 8531 raft_consensus.cc:729] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.824939 8529 raft_consensus.cc:374] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:23.825013 8529 raft_consensus.cc:729] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:23.825188 8529 consensus_queue.cc:260] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:23.825500 8529 ts_tablet_manager.cc:1398] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:23.824997 8531 consensus_queue.cc:260] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:23.825780 8531 ts_tablet_manager.cc:1398] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:49:23.909368 7970 raft_consensus.cc:482] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:23.909533 7970 raft_consensus.cc:504] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:23.909976 7970 leader_election.cc:288] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:23.910328 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "274ef2053af94bf1a038ec146c047063" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:23.910395 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "274ef2053af94bf1a038ec146c047063" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:23.910478 7011 raft_consensus.cc:2454] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:23.910554 6882 raft_consensus.cc:2454] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:23.910909 7208 leader_election.cc:302] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:23.911106 7970 raft_consensus.cc:2781] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:23.911159 7970 raft_consensus.cc:482] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:23.911185 7970 raft_consensus.cc:3037] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.912037 7970 raft_consensus.cc:504] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:23.912364 7970 leader_election.cc:288] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:23.912671 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "274ef2053af94bf1a038ec146c047063" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:23.912776 7011 raft_consensus.cc:3037] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.912750 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "274ef2053af94bf1a038ec146c047063" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:23.912868 6882 raft_consensus.cc:3037] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.913611 6882 raft_consensus.cc:2454] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:23.913615 7011 raft_consensus.cc:2454] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:23.913890 7208 leader_election.cc:302] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:23.914032 7970 raft_consensus.cc:2781] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:23.914116 7970 raft_consensus.cc:686] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:23.914269 7970 consensus_queue.cc:237] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:23.916013 6737 catalog_manager.cc:5484] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:23.926116 8526 raft_consensus.cc:482] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:23.926250 8526 raft_consensus.cc:504] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:23.926687 8526 leader_election.cc:288] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:23.927079 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69179df98341493987f1d70ab24f62ef" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:23.927114 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69179df98341493987f1d70ab24f62ef" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:23.927212 7274 raft_consensus.cc:2454] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:23.927235 7011 raft_consensus.cc:2454] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:23.927515 6817 leader_election.cc:302] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:23.927672 8526 raft_consensus.cc:2781] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:23.927724 8526 raft_consensus.cc:482] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:23.927767 8526 raft_consensus.cc:3037] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.928601 8526 raft_consensus.cc:504] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:23.928906 8526 leader_election.cc:288] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:23.929199 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69179df98341493987f1d70ab24f62ef" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:23.929186 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69179df98341493987f1d70ab24f62ef" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:23.929293 7274 raft_consensus.cc:3037] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.929296 7011 raft_consensus.cc:3037] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:23.930140 7011 raft_consensus.cc:2454] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:23.930243 7274 raft_consensus.cc:2454] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:23.930392 6815 leader_election.cc:302] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:23.930541 8526 raft_consensus.cc:2781] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:23.930624 8526 raft_consensus.cc:686] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:23.930802 8526 consensus_queue.cc:237] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:23.932663 6737 catalog_manager.cc:5484] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:24.186136 8572 raft_consensus.cc:482] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:24.186245 8572 raft_consensus.cc:504] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:24.186611 8572 leader_election.cc:288] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:24.186975 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f47b3eff1504b4d838c2afc3c0bb287" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:24.186985 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f47b3eff1504b4d838c2afc3c0bb287" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:24.187110 7274 raft_consensus.cc:2454] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:24.187110 6882 raft_consensus.cc:2454] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:24.187355 7079 leader_election.cc:302] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:24.187502 8572 raft_consensus.cc:2781] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:24.187556 8572 raft_consensus.cc:482] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:24.187602 8572 raft_consensus.cc:3037] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.188545 8572 raft_consensus.cc:504] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:24.188872 8572 leader_election.cc:288] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:24.189205 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f47b3eff1504b4d838c2afc3c0bb287" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:24.189188 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f47b3eff1504b4d838c2afc3c0bb287" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:24.189308 7274 raft_consensus.cc:3037] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.189308 6882 raft_consensus.cc:3037] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.190224 6882 raft_consensus.cc:2454] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:24.190224 7274 raft_consensus.cc:2454] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:24.190476 7079 leader_election.cc:302] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:24.190619 8572 raft_consensus.cc:2781] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:24.190704 8572 raft_consensus.cc:686] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:24.190865 8572 consensus_queue.cc:237] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:24.192606 6737 catalog_manager.cc:5484] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:24.309159 8584 consensus_queue.cc:1035] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.313378 8584 consensus_queue.cc:1035] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.326073 8526 consensus_queue.cc:1035] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.329376 8588 consensus_queue.cc:1035] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240426 09:49:24.376313 8598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:24.376606 8598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:24.379729 8598 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_10'
I20240426 09:49:24.501965 7274 raft_consensus.cc:1261] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:24.502012 6882 raft_consensus.cc:1261] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:24.502408 8575 consensus_queue.cc:1035] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.502578 8572 consensus_queue.cc:1035] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 112.295 ms
time per row: 0.0561474 ms
I20240426 09:49:24.537729 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:24.543092 6861 tablet_service.cc:1467] Processing CreateTablet for tablet b0a674c2acbe4f69a7b8fd30fa12b833 (DEFAULT_TABLE table=rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:24.543184 7123 tablet_service.cc:1467] Processing CreateTablet for tablet b0a674c2acbe4f69a7b8fd30fa12b833 (DEFAULT_TABLE table=rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:24.543452 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a674c2acbe4f69a7b8fd30fa12b833. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:24.543475 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a674c2acbe4f69a7b8fd30fa12b833. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:24.543629 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 948e1df8d52f42cebcd289fff377d6fb (DEFAULT_TABLE table=rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:24.543897 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948e1df8d52f42cebcd289fff377d6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:24.544303 6993 tablet_service.cc:1467] Processing CreateTablet for tablet b0a674c2acbe4f69a7b8fd30fa12b833 (DEFAULT_TABLE table=rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:24.544579 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a674c2acbe4f69a7b8fd30fa12b833. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:24.546301 8664 tablet_bootstrap.cc:492] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:24.546489 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 948e1df8d52f42cebcd289fff377d6fb (DEFAULT_TABLE table=rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:24.546715 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948e1df8d52f42cebcd289fff377d6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:24.546885 8666 tablet_bootstrap.cc:492] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:24.547057 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb0f5d2d55a45599c1b09c5684522e6 (DEFAULT_TABLE table=rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:24.547200 8664 tablet_bootstrap.cc:654] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:24.547313 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb0f5d2d55a45599c1b09c5684522e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:24.547804 8666 tablet_bootstrap.cc:654] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:24.549113 8665 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:24.549247 8666 tablet_bootstrap.cc:492] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:24.549331 8666 ts_tablet_manager.cc:1367] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:24.549736 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 948e1df8d52f42cebcd289fff377d6fb (DEFAULT_TABLE table=rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:24.549779 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb0f5d2d55a45599c1b09c5684522e6 (DEFAULT_TABLE table=rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:24.549803 8666 raft_consensus.cc:348] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:24.549902 8666 raft_consensus.cc:374] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:24.549937 8666 raft_consensus.cc:729] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:24.550020 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948e1df8d52f42cebcd289fff377d6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:24.550063 8666 consensus_queue.cc:260] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:24.550122 8665 tablet_bootstrap.cc:654] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:24.550359 8666 ts_tablet_manager.cc:1398] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:24.550518 8666 tablet_bootstrap.cc:492] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:24.550541 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb0f5d2d55a45599c1b09c5684522e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:24.551213 8665 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:24.551285 8665 ts_tablet_manager.cc:1367] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:24.551466 8666 tablet_bootstrap.cc:654] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:24.551753 8665 raft_consensus.cc:348] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:24.551856 8665 raft_consensus.cc:374] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:24.551883 8665 raft_consensus.cc:729] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:24.552006 8665 consensus_queue.cc:260] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:24.552290 8665 ts_tablet_manager.cc:1398] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:24.552485 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb0f5d2d55a45599c1b09c5684522e6 (DEFAULT_TABLE table=rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:24.552707 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb0f5d2d55a45599c1b09c5684522e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:24.554823 8665 tablet_bootstrap.cc:492] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:24.555543 8663 tablet_bootstrap.cc:492] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:24.555610 8666 tablet_bootstrap.cc:492] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:24.555692 8666 ts_tablet_manager.cc:1367] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:24.555869 8665 tablet_bootstrap.cc:654] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:24.556262 8666 raft_consensus.cc:348] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:24.556375 8666 raft_consensus.cc:374] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:24.556414 8666 raft_consensus.cc:729] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:24.556535 8666 consensus_queue.cc:260] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:24.556576 8663 tablet_bootstrap.cc:654] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:24.556800 8666 ts_tablet_manager.cc:1398] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:24.557749 8665 tablet_bootstrap.cc:492] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:24.557838 8665 ts_tablet_manager.cc:1367] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:24.558073 8664 tablet_bootstrap.cc:492] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:24.558159 8664 ts_tablet_manager.cc:1367] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240426 09:49:24.558283 8665 raft_consensus.cc:348] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:24.558375 8665 raft_consensus.cc:374] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:24.558405 8665 raft_consensus.cc:729] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:24.558635 8664 raft_consensus.cc:348] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:24.558789 8664 raft_consensus.cc:374] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:24.558830 8664 raft_consensus.cc:729] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:24.558808 8665 consensus_queue.cc:260] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:24.558955 8664 consensus_queue.cc:260] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:24.559134 8665 ts_tablet_manager.cc:1398] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:24.559221 8664 ts_tablet_manager.cc:1398] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:24.559443 8664 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:24.559648 8663 tablet_bootstrap.cc:492] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:24.559731 8663 ts_tablet_manager.cc:1367] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:24.560214 8663 raft_consensus.cc:348] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:24.560329 8663 raft_consensus.cc:374] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:24.560361 8663 raft_consensus.cc:729] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:24.560480 8663 consensus_queue.cc:260] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:24.560765 8664 tablet_bootstrap.cc:654] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:24.561376 8663 ts_tablet_manager.cc:1398] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:24.561522 8663 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:24.562427 8663 tablet_bootstrap.cc:654] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:24.564389 8664 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:24.564523 8664 ts_tablet_manager.cc:1367] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240426 09:49:24.564532 8663 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:24.564602 8663 ts_tablet_manager.cc:1367] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:24.565047 8664 raft_consensus.cc:348] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:24.565160 8664 raft_consensus.cc:374] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:24.565196 8664 raft_consensus.cc:729] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:24.565068 8663 raft_consensus.cc:348] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:24.565261 8663 raft_consensus.cc:374] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:24.565321 8663 raft_consensus.cc:729] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:24.565375 8664 consensus_queue.cc:260] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:24.565456 8663 consensus_queue.cc:260] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:24.565673 8664 ts_tablet_manager.cc:1398] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:24.565898 8664 tablet_bootstrap.cc:492] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:24.566355 8663 ts_tablet_manager.cc:1398] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:24.566982 8664 tablet_bootstrap.cc:654] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:24.570385 8664 tablet_bootstrap.cc:492] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:24.570472 8664 ts_tablet_manager.cc:1367] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:49:24.570895 8664 raft_consensus.cc:348] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:24.571024 8664 raft_consensus.cc:374] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:24.571067 8664 raft_consensus.cc:729] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:24.571183 8664 consensus_queue.cc:260] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:24.571455 8664 ts_tablet_manager.cc:1398] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:24.585383 8646 raft_consensus.cc:482] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:24.585500 8646 raft_consensus.cc:504] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:24.585880 8646 leader_election.cc:288] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:24.586179 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e1df8d52f42cebcd289fff377d6fb" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:24.586180 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e1df8d52f42cebcd289fff377d6fb" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:24.586315 7144 raft_consensus.cc:2454] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:24.586319 7275 raft_consensus.cc:2454] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:24.586606 6815 leader_election.cc:302] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:24.586803 8646 raft_consensus.cc:2781] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:24.586869 8646 raft_consensus.cc:482] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:24.586920 8646 raft_consensus.cc:3037] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.587755 8646 raft_consensus.cc:504] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:24.588075 8646 leader_election.cc:288] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:24.588343 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e1df8d52f42cebcd289fff377d6fb" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:24.588387 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "948e1df8d52f42cebcd289fff377d6fb" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:24.588459 7144 raft_consensus.cc:3037] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.588479 7275 raft_consensus.cc:3037] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.589360 7144 raft_consensus.cc:2454] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:24.589360 7275 raft_consensus.cc:2454] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:24.589640 6817 leader_election.cc:302] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:24.589790 8646 raft_consensus.cc:2781] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:24.589891 8646 raft_consensus.cc:686] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:24.590029 8646 consensus_queue.cc:237] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:24.591691 6737 catalog_manager.cc:5484] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:24.638151 8584 raft_consensus.cc:482] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:24.638298 8584 raft_consensus.cc:504] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:24.638698 8584 leader_election.cc:288] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:24.639117 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb0f5d2d55a45599c1b09c5684522e6" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:24.639222 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb0f5d2d55a45599c1b09c5684522e6" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:24.639273 7013 raft_consensus.cc:2454] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:24.639369 7144 raft_consensus.cc:2454] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:24.639652 7208 leader_election.cc:302] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:24.639817 8584 raft_consensus.cc:2781] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:24.639874 8584 raft_consensus.cc:482] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:24.639927 8584 raft_consensus.cc:3037] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.640831 8584 raft_consensus.cc:504] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:24.641153 8584 leader_election.cc:288] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:24.641523 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb0f5d2d55a45599c1b09c5684522e6" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:24.641522 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb0f5d2d55a45599c1b09c5684522e6" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:24.641657 7013 raft_consensus.cc:3037] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.641657 7144 raft_consensus.cc:3037] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.642575 7144 raft_consensus.cc:2454] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:24.642575 7013 raft_consensus.cc:2454] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:24.642858 7208 leader_election.cc:302] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:24.643121 8584 raft_consensus.cc:2781] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:24.643229 8584 raft_consensus.cc:686] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:24.643376 8584 consensus_queue.cc:237] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:24.645102 6733 catalog_manager.cc:5484] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:24.720325 8646 raft_consensus.cc:482] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:24.720443 8646 raft_consensus.cc:504] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:24.720821 8646 leader_election.cc:288] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:24.721218 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a674c2acbe4f69a7b8fd30fa12b833" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:24.721239 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a674c2acbe4f69a7b8fd30fa12b833" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:24.721343 7144 raft_consensus.cc:2454] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:24.721374 7013 raft_consensus.cc:2454] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:24.721632 6815 leader_election.cc:302] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:24.721926 8646 raft_consensus.cc:2781] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:24.721992 8646 raft_consensus.cc:482] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:24.722038 8646 raft_consensus.cc:3037] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.722878 8646 raft_consensus.cc:504] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:24.723142 8646 leader_election.cc:288] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:24.723428 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a674c2acbe4f69a7b8fd30fa12b833" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:24.723520 7144 raft_consensus.cc:3037] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.723507 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a674c2acbe4f69a7b8fd30fa12b833" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:24.723618 7011 raft_consensus.cc:3037] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:24.724256 7144 raft_consensus.cc:2454] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:24.724459 7011 raft_consensus.cc:2454] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:24.724467 6815 leader_election.cc:302] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:24.724646 8646 raft_consensus.cc:2781] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:24.724752 8646 raft_consensus.cc:686] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:24.724889 8646 consensus_queue.cc:237] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:24.726644 6733 catalog_manager.cc:5484] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
W20240426 09:49:24.843804 8679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:24.844039 8679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:24.846921 8679 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_11'
I20240426 09:49:24.925083 7275 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:24.925375 7011 raft_consensus.cc:1261] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:24.925549 8526 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.926714 7144 raft_consensus.cc:1261] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:24.928022 7144 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:24.928907 8603 mvcc.cc:205] Tried to move back new op lower bound from 7021055856326541312 to 7021055854962003968. Current Snapshot: MvccSnapshot[applied={T|T < 7021055856326541312}]
I20240426 09:49:24.929157 8588 consensus_queue.cc:1035] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.929459 8588 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.929731 8588 consensus_queue.cc:1035] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.942296 7013 raft_consensus.cc:1261] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:24.944223 8705 consensus_queue.cc:1035] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.944283 7144 raft_consensus.cc:1261] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:24.945276 8705 consensus_queue.cc:1035] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:24.960649 8600 mvcc.cc:205] Tried to move back new op lower bound from 7021055856326541312 to 7021055854962003968. Current Snapshot: MvccSnapshot[applied={T|T < 7021055856326541312}]
INSERT report
rows total: 2000
time total: 109.131 ms
time per row: 0.0545657 ms
I20240426 09:49:24.998461 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:25.004155 6991 tablet_service.cc:1467] Processing CreateTablet for tablet c3dc11eb66744cb496ffbc2ff80ad571 (DEFAULT_TABLE table=rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.004153 6861 tablet_service.cc:1467] Processing CreateTablet for tablet c3dc11eb66744cb496ffbc2ff80ad571 (DEFAULT_TABLE table=rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.004472 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3dc11eb66744cb496ffbc2ff80ad571. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.004470 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3dc11eb66744cb496ffbc2ff80ad571. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.005040 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 19229a19d57c489fa224f5ce6a439485 (DEFAULT_TABLE table=rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.005167 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 19229a19d57c489fa224f5ce6a439485 (DEFAULT_TABLE table=rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.005249 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19229a19d57c489fa224f5ce6a439485. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.005417 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19229a19d57c489fa224f5ce6a439485. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.006675 8666 tablet_bootstrap.cc:492] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:25.007334 7123 tablet_service.cc:1467] Processing CreateTablet for tablet f3b2b3e1d02b49269d19d284cab418da (DEFAULT_TABLE table=rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.007565 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b2b3e1d02b49269d19d284cab418da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.007582 8666 tablet_bootstrap.cc:654] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.007946 7254 tablet_service.cc:1467] Processing CreateTablet for tablet f3b2b3e1d02b49269d19d284cab418da (DEFAULT_TABLE table=rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.007946 7253 tablet_service.cc:1467] Processing CreateTablet for tablet c3dc11eb66744cb496ffbc2ff80ad571 (DEFAULT_TABLE table=rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.008266 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3dc11eb66744cb496ffbc2ff80ad571. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.008435 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b2b3e1d02b49269d19d284cab418da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.008698 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 19229a19d57c489fa224f5ce6a439485 (DEFAULT_TABLE table=rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.008924 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19229a19d57c489fa224f5ce6a439485. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.009382 8663 tablet_bootstrap.cc:492] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:25.010098 8663 tablet_bootstrap.cc:654] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.010444 8665 tablet_bootstrap.cc:492] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:25.010569 8664 tablet_bootstrap.cc:492] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:25.011291 8665 tablet_bootstrap.cc:654] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.011358 8664 tablet_bootstrap.cc:654] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.012043 6862 tablet_service.cc:1467] Processing CreateTablet for tablet f3b2b3e1d02b49269d19d284cab418da (DEFAULT_TABLE table=rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.012262 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b2b3e1d02b49269d19d284cab418da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.012328 8665 tablet_bootstrap.cc:492] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:25.012391 8665 ts_tablet_manager.cc:1367] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:25.012840 8665 raft_consensus.cc:348] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:25.012938 8665 raft_consensus.cc:374] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.012967 8665 raft_consensus.cc:729] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.013124 8665 consensus_queue.cc:260] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:25.013444 8665 ts_tablet_manager.cc:1398] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.013598 8665 tablet_bootstrap.cc:492] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:25.014488 8665 tablet_bootstrap.cc:654] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.014802 8663 tablet_bootstrap.cc:492] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:25.014890 8663 ts_tablet_manager.cc:1367] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240426 09:49:25.015314 8663 raft_consensus.cc:348] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.015420 8663 raft_consensus.cc:374] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.015451 8663 raft_consensus.cc:729] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.015574 8663 consensus_queue.cc:260] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.015650 8666 tablet_bootstrap.cc:492] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:25.015734 8666 ts_tablet_manager.cc:1367] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240426 09:49:25.015867 8663 ts_tablet_manager.cc:1398] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.016059 8663 tablet_bootstrap.cc:492] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:25.016180 8666 raft_consensus.cc:348] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.016273 8666 raft_consensus.cc:374] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.016301 8666 raft_consensus.cc:729] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.016412 8666 consensus_queue.cc:260] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.016649 8666 ts_tablet_manager.cc:1398] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.016793 8666 tablet_bootstrap.cc:492] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:25.017691 8666 tablet_bootstrap.cc:654] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.018090 8664 tablet_bootstrap.cc:492] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:25.018185 8664 ts_tablet_manager.cc:1367] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:49:25.018682 8664 raft_consensus.cc:348] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.018911 8664 raft_consensus.cc:374] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.018870 8666 tablet_bootstrap.cc:492] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:25.018962 8664 raft_consensus.cc:729] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.019078 8666 ts_tablet_manager.cc:1367] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:25.019094 8664 consensus_queue.cc:260] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.019646 8664 ts_tablet_manager.cc:1398] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:25.019645 8666 raft_consensus.cc:348] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.019827 8666 raft_consensus.cc:374] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.019904 8666 raft_consensus.cc:729] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.020066 8666 consensus_queue.cc:260] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.020453 8664 tablet_bootstrap.cc:492] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:25.020561 8666 ts_tablet_manager.cc:1398] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:25.021358 8664 tablet_bootstrap.cc:654] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.017202 8663 tablet_bootstrap.cc:654] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.022312 8665 tablet_bootstrap.cc:492] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:25.022542 8665 ts_tablet_manager.cc:1367] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240426 09:49:25.023197 8665 raft_consensus.cc:348] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.023312 8665 raft_consensus.cc:374] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.023391 8665 raft_consensus.cc:729] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.023761 8665 consensus_queue.cc:260] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.023829 8664 tablet_bootstrap.cc:492] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:25.023897 8664 ts_tablet_manager.cc:1367] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:25.024291 8664 raft_consensus.cc:348] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:25.024394 8664 raft_consensus.cc:374] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.024430 8664 raft_consensus.cc:729] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.024544 8664 consensus_queue.cc:260] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:25.024801 8664 ts_tablet_manager.cc:1398] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.024823 8665 ts_tablet_manager.cc:1398] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:25.025715 8663 tablet_bootstrap.cc:492] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:25.025815 8663 ts_tablet_manager.cc:1367] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:25.026304 8663 raft_consensus.cc:348] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.026407 8663 raft_consensus.cc:374] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.026443 8663 raft_consensus.cc:729] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.026559 8663 consensus_queue.cc:260] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.026830 8663 ts_tablet_manager.cc:1398] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.027050 8663 tablet_bootstrap.cc:492] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:25.027727 8663 tablet_bootstrap.cc:654] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.028805 8663 tablet_bootstrap.cc:492] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:25.028887 8663 ts_tablet_manager.cc:1367] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:25.029215 8663 raft_consensus.cc:348] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:25.029301 8663 raft_consensus.cc:374] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.029340 8663 raft_consensus.cc:729] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.029454 8663 consensus_queue.cc:260] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:25.029711 8663 ts_tablet_manager.cc:1398] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:25.067375 8646 raft_consensus.cc:482] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:25.067526 8646 raft_consensus.cc:504] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.068024 8646 leader_election.cc:288] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:25.068387 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3dc11eb66744cb496ffbc2ff80ad571" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:25.068372 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3dc11eb66744cb496ffbc2ff80ad571" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:25.068522 7013 raft_consensus.cc:2454] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:25.068523 7275 raft_consensus.cc:2454] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:25.068815 6817 leader_election.cc:302] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:25.069027 8646 raft_consensus.cc:2781] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:25.069087 8646 raft_consensus.cc:482] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:25.069113 8646 raft_consensus.cc:3037] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.070036 8646 raft_consensus.cc:504] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.070359 8646 leader_election.cc:288] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:25.070709 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3dc11eb66744cb496ffbc2ff80ad571" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:25.070753 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3dc11eb66744cb496ffbc2ff80ad571" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:25.070863 7275 raft_consensus.cc:3037] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.070922 7013 raft_consensus.cc:3037] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.071679 7275 raft_consensus.cc:2454] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:25.071696 7013 raft_consensus.cc:2454] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:25.071990 6815 leader_election.cc:302] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:25.072160 8646 raft_consensus.cc:2781] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:25.072261 8646 raft_consensus.cc:686] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:25.072402 8646 consensus_queue.cc:237] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.074152 6733 catalog_manager.cc:5484] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:25.142627 8606 raft_consensus.cc:482] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:25.142845 8606 raft_consensus.cc:504] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.143221 8606 leader_election.cc:288] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:25.143620 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19229a19d57c489fa224f5ce6a439485" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:25.143663 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19229a19d57c489fa224f5ce6a439485" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:25.143788 6882 raft_consensus.cc:2454] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:25.143788 7143 raft_consensus.cc:2454] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:25.144117 6948 leader_election.cc:302] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:25.145666 8606 raft_consensus.cc:2781] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:25.145722 8606 raft_consensus.cc:482] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:25.145747 8606 raft_consensus.cc:3037] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.146592 8606 raft_consensus.cc:504] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.146996 8606 leader_election.cc:288] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:25.147344 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19229a19d57c489fa224f5ce6a439485" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:25.147361 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19229a19d57c489fa224f5ce6a439485" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:25.147464 6882 raft_consensus.cc:3037] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.147478 7143 raft_consensus.cc:3037] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.148260 7143 raft_consensus.cc:2454] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:25.148402 6882 raft_consensus.cc:2454] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:25.148682 6946 leader_election.cc:302] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:25.148862 8606 raft_consensus.cc:2781] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:25.149041 8606 raft_consensus.cc:686] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:25.149240 8606 consensus_queue.cc:237] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.151522 6733 catalog_manager.cc:5484] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:25.166309 8705 raft_consensus.cc:482] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:25.166482 8705 raft_consensus.cc:504] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:25.166949 8705 leader_election.cc:288] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:25.167323 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b2b3e1d02b49269d19d284cab418da" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:25.167352 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b2b3e1d02b49269d19d284cab418da" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:25.167465 7143 raft_consensus.cc:2454] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:25.167465 6882 raft_consensus.cc:2454] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:25.167750 7210 leader_election.cc:302] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:25.167901 8705 raft_consensus.cc:2781] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:25.167968 8705 raft_consensus.cc:482] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:25.168013 8705 raft_consensus.cc:3037] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.169435 8705 raft_consensus.cc:504] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:25.170076 8705 leader_election.cc:288] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:25.170037 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b2b3e1d02b49269d19d284cab418da" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:25.170158 7143 raft_consensus.cc:3037] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.170372 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3b2b3e1d02b49269d19d284cab418da" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:25.170500 6882 raft_consensus.cc:3037] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.171248 7143 raft_consensus.cc:2454] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:25.171538 6882 raft_consensus.cc:2454] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:25.171533 7208 leader_election.cc:302] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:25.171696 8705 raft_consensus.cc:2781] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:25.172078 8705 raft_consensus.cc:686] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:25.172207 8705 consensus_queue.cc:237] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:25.174181 6733 catalog_manager.cc:5484] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
W20240426 09:49:25.315799 8725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:25.316118 8725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:25.319183 8725 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_12'
I20240426 09:49:25.408443 7143 raft_consensus.cc:1261] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:25.408903 8722 consensus_queue.cc:1035] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:25.410352 6882 raft_consensus.cc:1261] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:25.411084 7144 raft_consensus.cc:1261] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:25.411293 6882 raft_consensus.cc:1261] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:25.411445 7013 raft_consensus.cc:1261] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:25.412509 8584 consensus_queue.cc:1035] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:25.413410 8584 consensus_queue.cc:1035] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:25.413923 7275 raft_consensus.cc:1261] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:25.414410 8646 consensus_queue.cc:1035] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:25.415083 8606 consensus_queue.cc:1035] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:25.414687 8646 consensus_queue.cc:1035] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:25.422412 8601 mvcc.cc:205] Tried to move back new op lower bound from 7021055858309054464 to 7021055857253515264. Current Snapshot: MvccSnapshot[applied={T|T < 7021055858309054464}]
INSERT report
rows total: 2000
time total: 116.488 ms
time per row: 0.0582441 ms
I20240426 09:49:25.479239 6733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:25.487206 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 643fd2d359a14f8eaa8d8a0d0879ef61 (DEFAULT_TABLE table=rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.487318 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 643fd2d359a14f8eaa8d8a0d0879ef61 (DEFAULT_TABLE table=rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.487530 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 643fd2d359a14f8eaa8d8a0d0879ef61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.487620 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 643fd2d359a14f8eaa8d8a0d0879ef61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.488215 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 643fd2d359a14f8eaa8d8a0d0879ef61 (DEFAULT_TABLE table=rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.488245 6862 tablet_service.cc:1467] Processing CreateTablet for tablet ce1e72dc91434c4ab63639fcc50da2ec (DEFAULT_TABLE table=rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.488462 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 643fd2d359a14f8eaa8d8a0d0879ef61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.488500 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce1e72dc91434c4ab63639fcc50da2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.489892 7255 tablet_service.cc:1467] Processing CreateTablet for tablet ce1e72dc91434c4ab63639fcc50da2ec (DEFAULT_TABLE table=rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.490041 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 11a10ff62d224cf49fcb71d11b029537 (DEFAULT_TABLE table=rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.490247 8664 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:25.490551 8666 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:25.490669 8663 tablet_bootstrap.cc:492] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:25.491279 8664 tablet_bootstrap.cc:654] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.491343 8666 tablet_bootstrap.cc:654] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.491654 8663 tablet_bootstrap.cc:654] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.492007 8665 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:25.490109 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce1e72dc91434c4ab63639fcc50da2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.492736 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 11a10ff62d224cf49fcb71d11b029537 (DEFAULT_TABLE table=rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.492965 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a10ff62d224cf49fcb71d11b029537. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.493366 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a10ff62d224cf49fcb71d11b029537. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.493417 8663 tablet_bootstrap.cc:492] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:25.493494 8663 ts_tablet_manager.cc:1367] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:25.493966 8663 raft_consensus.cc:348] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.494068 8663 raft_consensus.cc:374] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.494097 8663 raft_consensus.cc:729] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.494212 8663 consensus_queue.cc:260] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.494493 8663 ts_tablet_manager.cc:1398] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.494832 6991 tablet_service.cc:1467] Processing CreateTablet for tablet ce1e72dc91434c4ab63639fcc50da2ec (DEFAULT_TABLE table=rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.495052 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce1e72dc91434c4ab63639fcc50da2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.496501 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 11a10ff62d224cf49fcb71d11b029537 (DEFAULT_TABLE table=rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:25.496910 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a10ff62d224cf49fcb71d11b029537. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:25.497498 8664 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:25.497583 8664 ts_tablet_manager.cc:1367] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:25.497599 8665 tablet_bootstrap.cc:654] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.498024 8664 raft_consensus.cc:348] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.498129 8664 raft_consensus.cc:374] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.498167 8664 raft_consensus.cc:729] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.498296 8664 consensus_queue.cc:260] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.498576 8664 ts_tablet_manager.cc:1398] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.498589 8665 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:25.498677 8665 ts_tablet_manager.cc:1367] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:25.499087 8665 raft_consensus.cc:348] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.499199 8665 raft_consensus.cc:374] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.499236 8665 raft_consensus.cc:729] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.499317 8666 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:25.499361 8665 consensus_queue.cc:260] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.499413 8666 ts_tablet_manager.cc:1367] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240426 09:49:25.499644 8665 ts_tablet_manager.cc:1398] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.499835 8665 tablet_bootstrap.cc:492] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:25.499892 8666 raft_consensus.cc:348] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.499987 8666 raft_consensus.cc:374] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.500021 8666 raft_consensus.cc:729] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.500142 8666 consensus_queue.cc:260] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.500439 8666 ts_tablet_manager.cc:1398] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.501001 8666 tablet_bootstrap.cc:492] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:25.501816 8666 tablet_bootstrap.cc:654] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.502856 8666 tablet_bootstrap.cc:492] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:25.502940 8666 ts_tablet_manager.cc:1367] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:25.503355 8666 raft_consensus.cc:348] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.503459 8666 raft_consensus.cc:374] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.503489 8666 raft_consensus.cc:729] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.503638 8666 consensus_queue.cc:260] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.503937 8665 tablet_bootstrap.cc:654] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.503962 8666 ts_tablet_manager.cc:1398] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.504107 8666 tablet_bootstrap.cc:492] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:25.504894 8666 tablet_bootstrap.cc:654] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.505055 8665 tablet_bootstrap.cc:492] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:25.505182 8665 ts_tablet_manager.cc:1367] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:25.505820 8665 raft_consensus.cc:348] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.505957 8665 raft_consensus.cc:374] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.505998 8665 raft_consensus.cc:729] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.506119 8665 consensus_queue.cc:260] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.506177 8664 tablet_bootstrap.cc:492] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:25.506434 8665 ts_tablet_manager.cc:1398] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.506610 8665 tablet_bootstrap.cc:492] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:25.507102 8664 tablet_bootstrap.cc:654] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.507505 8665 tablet_bootstrap.cc:654] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:25.508473 8665 tablet_bootstrap.cc:492] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:25.508548 8665 ts_tablet_manager.cc:1367] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:25.508942 8665 raft_consensus.cc:348] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.509054 8665 raft_consensus.cc:374] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.509090 8665 raft_consensus.cc:729] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.509192 8664 tablet_bootstrap.cc:492] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:25.509207 8665 consensus_queue.cc:260] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.509272 8664 ts_tablet_manager.cc:1367] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:25.509485 8665 ts_tablet_manager.cc:1398] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.509704 8664 raft_consensus.cc:348] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.509821 8664 raft_consensus.cc:374] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.509855 8664 raft_consensus.cc:729] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.509966 8664 consensus_queue.cc:260] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.510236 8664 ts_tablet_manager.cc:1398] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.511683 8666 tablet_bootstrap.cc:492] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:25.511785 8666 ts_tablet_manager.cc:1367] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:49:25.512253 8666 raft_consensus.cc:348] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.512365 8666 raft_consensus.cc:374] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:25.512398 8666 raft_consensus.cc:729] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:25.512535 8666 consensus_queue.cc:260] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.512825 8666 ts_tablet_manager.cc:1398] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:25.514807 8606 raft_consensus.cc:482] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:25.514887 8606 raft_consensus.cc:504] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.515201 8606 leader_election.cc:288] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:25.515516 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a10ff62d224cf49fcb71d11b029537" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:25.515627 7144 raft_consensus.cc:2454] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:25.515937 6946 leader_election.cc:302] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:25.515921 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a10ff62d224cf49fcb71d11b029537" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:25.516022 7274 raft_consensus.cc:2454] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:25.516081 8606 raft_consensus.cc:2781] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:25.516124 8606 raft_consensus.cc:482] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:25.516165 8606 raft_consensus.cc:3037] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.516894 8606 raft_consensus.cc:504] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.517184 8606 leader_election.cc:288] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:25.517427 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a10ff62d224cf49fcb71d11b029537" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:25.517513 7144 raft_consensus.cc:3037] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.517493 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11a10ff62d224cf49fcb71d11b029537" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:25.517580 7274 raft_consensus.cc:3037] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.518159 7144 raft_consensus.cc:2454] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:25.518455 7274 raft_consensus.cc:2454] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:25.518469 6946 leader_election.cc:302] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:25.518687 8606 raft_consensus.cc:2781] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:25.518805 8606 raft_consensus.cc:686] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:25.518940 8606 consensus_queue.cc:237] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.520535 6735 catalog_manager.cc:5484] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:25.623759 8584 raft_consensus.cc:482] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:25.623894 8584 raft_consensus.cc:504] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.624256 8584 leader_election.cc:288] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:25.624550 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce1e72dc91434c4ab63639fcc50da2ec" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:25.624591 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce1e72dc91434c4ab63639fcc50da2ec" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:25.624661 6880 raft_consensus.cc:2454] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:25.624742 7013 raft_consensus.cc:2454] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:25.624918 7210 leader_election.cc:302] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:25.625056 8584 raft_consensus.cc:2781] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:25.625111 8584 raft_consensus.cc:482] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:25.625149 8584 raft_consensus.cc:3037] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.626013 8584 raft_consensus.cc:504] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.626318 8584 leader_election.cc:288] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:25.626621 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce1e72dc91434c4ab63639fcc50da2ec" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:25.626720 6880 raft_consensus.cc:3037] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.627475 6880 raft_consensus.cc:2454] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:25.627687 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce1e72dc91434c4ab63639fcc50da2ec" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:25.627734 7210 leader_election.cc:302] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:25.627815 7013 raft_consensus.cc:3037] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.627884 8584 raft_consensus.cc:2781] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:25.627975 8584 raft_consensus.cc:686] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:25.628078 8584 consensus_queue.cc:237] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:25.628612 7013 raft_consensus.cc:2454] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:25.629796 6735 catalog_manager.cc:5484] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:25.827644 8584 raft_consensus.cc:482] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:25.827771 8584 raft_consensus.cc:504] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.828140 8584 leader_election.cc:288] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:25.828473 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643fd2d359a14f8eaa8d8a0d0879ef61" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:25.828627 7144 raft_consensus.cc:2454] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:25.828593 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643fd2d359a14f8eaa8d8a0d0879ef61" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:25.828720 7013 raft_consensus.cc:2454] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:25.828949 7208 leader_election.cc:302] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:25.829109 8584 raft_consensus.cc:2781] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:25.829161 8584 raft_consensus.cc:482] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:25.829207 8584 raft_consensus.cc:3037] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.830022 8584 raft_consensus.cc:504] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.830313 8584 leader_election.cc:288] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:25.830655 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643fd2d359a14f8eaa8d8a0d0879ef61" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:25.830662 7144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643fd2d359a14f8eaa8d8a0d0879ef61" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:25.830806 7144 raft_consensus.cc:3037] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.830811 7013 raft_consensus.cc:3037] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:25.831643 7013 raft_consensus.cc:2454] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:25.831744 7144 raft_consensus.cc:2454] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:25.831957 7208 leader_election.cc:302] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:25.832115 8584 raft_consensus.cc:2781] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:25.832208 8584 raft_consensus.cc:686] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:25.832340 8584 consensus_queue.cc:237] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:25.834064 6735 catalog_manager.cc:5484] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
W20240426 09:49:25.980291 8781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:25.980548 8781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:25.983655 8781 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_13'
I20240426 09:49:26.010807 8606 consensus_queue.cc:1035] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.013801 8722 consensus_queue.cc:1035] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.073055 7011 raft_consensus.cc:1261] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.073504 6880 raft_consensus.cc:1261] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.073491 8705 consensus_queue.cc:1035] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.073954 7011 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.074117 8705 consensus_queue.cc:1035] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.075330 8584 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.077709 7143 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.080832 8584 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 107.388 ms
time per row: 0.053694 ms
I20240426 09:49:26.136472 6735 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:26.143646 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc287ff7ea640a38537c74e9ff156f7 (DEFAULT_TABLE table=rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.143712 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc287ff7ea640a38537c74e9ff156f7 (DEFAULT_TABLE table=rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.144012 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc287ff7ea640a38537c74e9ff156f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.144030 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc287ff7ea640a38537c74e9ff156f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.146561 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc287ff7ea640a38537c74e9ff156f7 (DEFAULT_TABLE table=rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.146865 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc287ff7ea640a38537c74e9ff156f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.147332 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 9ed2745f48d141c9a89ae8e13606a782 (DEFAULT_TABLE table=rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.147559 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ed2745f48d141c9a89ae8e13606a782. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.147526 6861 tablet_service.cc:1467] Processing CreateTablet for tablet b85b6cfec5314d75aebe17670edf8d04 (DEFAULT_TABLE table=rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.147603 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 9ed2745f48d141c9a89ae8e13606a782 (DEFAULT_TABLE table=rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.147735 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b85b6cfec5314d75aebe17670edf8d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.147874 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ed2745f48d141c9a89ae8e13606a782. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.149582 8841 tablet_bootstrap.cc:492] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:26.150233 8843 tablet_bootstrap.cc:492] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:26.150619 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 9ed2745f48d141c9a89ae8e13606a782 (DEFAULT_TABLE table=rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.150676 8841 tablet_bootstrap.cc:654] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.150874 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ed2745f48d141c9a89ae8e13606a782. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.150852 6993 tablet_service.cc:1467] Processing CreateTablet for tablet b85b6cfec5314d75aebe17670edf8d04 (DEFAULT_TABLE table=rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.151110 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b85b6cfec5314d75aebe17670edf8d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.152897 7122 tablet_service.cc:1467] Processing CreateTablet for tablet b85b6cfec5314d75aebe17670edf8d04 (DEFAULT_TABLE table=rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.153160 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b85b6cfec5314d75aebe17670edf8d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.153836 8843 tablet_bootstrap.cc:654] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.155057 8843 tablet_bootstrap.cc:492] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:26.155191 8843 ts_tablet_manager.cc:1367] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:26.155558 8843 raft_consensus.cc:348] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:26.155650 8843 raft_consensus.cc:374] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.155681 8843 raft_consensus.cc:729] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.155879 8843 consensus_queue.cc:260] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:26.156401 8843 ts_tablet_manager.cc:1398] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:26.156603 8843 tablet_bootstrap.cc:492] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:26.157450 8843 tablet_bootstrap.cc:654] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.158552 8843 tablet_bootstrap.cc:492] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:26.158663 8843 ts_tablet_manager.cc:1367] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:26.159026 8844 tablet_bootstrap.cc:492] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:26.159181 8843 raft_consensus.cc:348] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.159329 8843 raft_consensus.cc:374] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.159400 8843 raft_consensus.cc:729] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.159556 8843 consensus_queue.cc:260] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.159927 8843 ts_tablet_manager.cc:1398] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:26.160111 8843 tablet_bootstrap.cc:492] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:26.160704 8844 tablet_bootstrap.cc:654] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.160977 8843 tablet_bootstrap.cc:654] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.161561 8842 tablet_bootstrap.cc:492] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:26.162318 8842 tablet_bootstrap.cc:654] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.163235 8842 tablet_bootstrap.cc:492] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:26.163339 8841 tablet_bootstrap.cc:492] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:26.163338 8844 tablet_bootstrap.cc:492] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:26.163437 8841 ts_tablet_manager.cc:1367] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240426 09:49:26.163439 8844 ts_tablet_manager.cc:1367] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:26.163440 8842 ts_tablet_manager.cc:1367] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:26.163913 8842 raft_consensus.cc:348] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:26.163959 8841 raft_consensus.cc:348] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:26.164081 8841 raft_consensus.cc:374] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.164124 8841 raft_consensus.cc:729] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.164108 8842 raft_consensus.cc:374] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.164245 8842 raft_consensus.cc:729] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.164265 8841 consensus_queue.cc:260] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:26.164465 8842 consensus_queue.cc:260] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:26.164575 8841 ts_tablet_manager.cc:1398] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:26.164795 8841 tablet_bootstrap.cc:492] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:26.164932 8842 ts_tablet_manager.cc:1398] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:26.165164 8842 tablet_bootstrap.cc:492] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:26.165683 8841 tablet_bootstrap.cc:654] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.165947 8842 tablet_bootstrap.cc:654] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.167155 8844 raft_consensus.cc:348] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.167325 8844 raft_consensus.cc:374] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.167402 8844 raft_consensus.cc:729] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.167570 8844 consensus_queue.cc:260] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.167953 8844 ts_tablet_manager.cc:1398] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:26.168344 8843 tablet_bootstrap.cc:492] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:26.168422 8843 ts_tablet_manager.cc:1367] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:26.168789 8843 raft_consensus.cc:348] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.168876 8843 raft_consensus.cc:374] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.168901 8843 raft_consensus.cc:729] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.168999 8843 consensus_queue.cc:260] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.169226 8843 ts_tablet_manager.cc:1398] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:26.171631 8841 tablet_bootstrap.cc:492] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:26.171648 8842 tablet_bootstrap.cc:492] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:26.171716 8842 ts_tablet_manager.cc:1367] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:26.171720 8841 ts_tablet_manager.cc:1367] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:26.172132 8841 raft_consensus.cc:348] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.172236 8841 raft_consensus.cc:374] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.172278 8841 raft_consensus.cc:729] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.172384 8841 consensus_queue.cc:260] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.172654 8841 ts_tablet_manager.cc:1398] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:26.172629 8842 raft_consensus.cc:348] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.172729 8842 raft_consensus.cc:374] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.172766 8842 raft_consensus.cc:729] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.172884 8842 consensus_queue.cc:260] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.173225 8842 ts_tablet_manager.cc:1398] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:26.173367 8842 tablet_bootstrap.cc:492] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:26.174490 8842 tablet_bootstrap.cc:654] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.181509 8842 tablet_bootstrap.cc:492] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:26.181593 8842 ts_tablet_manager.cc:1367] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:49:26.181903 8842 raft_consensus.cc:348] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.181988 8842 raft_consensus.cc:374] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.182010 8842 raft_consensus.cc:729] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.182134 8842 consensus_queue.cc:260] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.182395 8842 ts_tablet_manager.cc:1398] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:26.190223 8606 raft_consensus.cc:482] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:26.190331 8606 raft_consensus.cc:504] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.190788 8606 leader_election.cc:288] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:26.191130 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b85b6cfec5314d75aebe17670edf8d04" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:26.191154 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b85b6cfec5314d75aebe17670edf8d04" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:26.191265 6880 raft_consensus.cc:2454] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:26.191304 7143 raft_consensus.cc:2454] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:26.191537 6948 leader_election.cc:302] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:26.191748 8606 raft_consensus.cc:2781] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:26.191802 8606 raft_consensus.cc:482] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:26.191828 8606 raft_consensus.cc:3037] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.192595 8606 raft_consensus.cc:504] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.192909 8606 leader_election.cc:288] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:26.193234 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b85b6cfec5314d75aebe17670edf8d04" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:26.193235 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b85b6cfec5314d75aebe17670edf8d04" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:26.193354 6880 raft_consensus.cc:3037] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.193356 7143 raft_consensus.cc:3037] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.194309 6880 raft_consensus.cc:2454] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:26.194309 7143 raft_consensus.cc:2454] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:26.194620 6948 leader_election.cc:302] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:26.194794 8606 raft_consensus.cc:2781] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:26.194913 8606 raft_consensus.cc:686] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:26.195062 8606 consensus_queue.cc:237] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.196766 6735 catalog_manager.cc:5484] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:26.264683 8809 raft_consensus.cc:482] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:26.264845 8809 raft_consensus.cc:504] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.265266 8809 leader_election.cc:288] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:26.265726 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ed2745f48d141c9a89ae8e13606a782" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:26.265726 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ed2745f48d141c9a89ae8e13606a782" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:26.265892 7275 raft_consensus.cc:2454] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:26.265892 7143 raft_consensus.cc:2454] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:26.266188 6817 leader_election.cc:302] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:26.266355 8809 raft_consensus.cc:2781] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:26.266419 8809 raft_consensus.cc:482] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:26.266445 8809 raft_consensus.cc:3037] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.267446 8809 raft_consensus.cc:504] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.267791 8809 leader_election.cc:288] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:26.268074 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ed2745f48d141c9a89ae8e13606a782" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:26.268193 7143 raft_consensus.cc:3037] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.268179 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ed2745f48d141c9a89ae8e13606a782" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:26.268288 7275 raft_consensus.cc:3037] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.268950 7143 raft_consensus.cc:2454] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:26.269069 7275 raft_consensus.cc:2454] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:26.269183 6815 leader_election.cc:302] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:26.269342 8809 raft_consensus.cc:2781] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:26.269433 8809 raft_consensus.cc:686] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:26.269573 8809 consensus_queue.cc:237] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:26.271585 6735 catalog_manager.cc:5484] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:26.320952 8809 raft_consensus.cc:482] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:26.321089 8809 raft_consensus.cc:504] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:26.321449 8809 leader_election.cc:288] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:26.321942 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc287ff7ea640a38537c74e9ff156f7" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:26.321940 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc287ff7ea640a38537c74e9ff156f7" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:26.322073 7143 raft_consensus.cc:2454] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:26.322079 7013 raft_consensus.cc:2454] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:26.322388 6815 leader_election.cc:302] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:26.322542 8809 raft_consensus.cc:2781] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:26.322598 8809 raft_consensus.cc:482] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:26.322638 8809 raft_consensus.cc:3037] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.323510 8809 raft_consensus.cc:504] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:26.323823 8809 leader_election.cc:288] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:26.324141 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc287ff7ea640a38537c74e9ff156f7" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:26.324177 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc287ff7ea640a38537c74e9ff156f7" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:26.324263 7143 raft_consensus.cc:3037] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.324270 7013 raft_consensus.cc:3037] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.325042 7143 raft_consensus.cc:2454] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:26.325189 7013 raft_consensus.cc:2454] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:26.325387 6815 leader_election.cc:302] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:26.325632 8809 raft_consensus.cc:2781] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:26.325714 8809 raft_consensus.cc:686] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:26.325860 8809 consensus_queue.cc:237] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:26.327800 6735 catalog_manager.cc:5484] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
W20240426 09:49:26.448949 8857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:26.449306 8857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:26.452569 8857 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_14'
I20240426 09:49:26.536697 7013 raft_consensus.cc:1261] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.537142 8809 consensus_queue.cc:1035] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.539395 7143 raft_consensus.cc:1261] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.539584 7275 raft_consensus.cc:1261] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.539911 6880 raft_consensus.cc:1261] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.540048 7143 raft_consensus.cc:1261] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.540500 8606 consensus_queue.cc:1035] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.540702 8722 consensus_queue.cc:1035] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.541287 8764 consensus_queue.cc:1035] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.541776 8764 consensus_queue.cc:1035] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.549594 7144 raft_consensus.cc:1261] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.550565 8809 consensus_queue.cc:1035] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 117.377 ms
time per row: 0.0586886 ms
I20240426 09:49:26.612674 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:26.617985 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 7e130e7733534aedb221d7772a2d29ce (DEFAULT_TABLE table=rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.618060 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 7e130e7733534aedb221d7772a2d29ce (DEFAULT_TABLE table=rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.618324 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e130e7733534aedb221d7772a2d29ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.618324 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e130e7733534aedb221d7772a2d29ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.618261 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 7e130e7733534aedb221d7772a2d29ce (DEFAULT_TABLE table=rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.618541 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e130e7733534aedb221d7772a2d29ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.620352 8844 tablet_bootstrap.cc:492] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:26.620432 8842 tablet_bootstrap.cc:492] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:26.621249 8844 tablet_bootstrap.cc:654] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.621251 8842 tablet_bootstrap.cc:654] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.621887 8841 tablet_bootstrap.cc:492] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:26.622288 8844 tablet_bootstrap.cc:492] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:26.622380 8844 ts_tablet_manager.cc:1367] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:26.622845 8841 tablet_bootstrap.cc:654] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.622860 8844 raft_consensus.cc:348] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.622972 8844 raft_consensus.cc:374] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.623006 8844 raft_consensus.cc:729] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.623142 8844 consensus_queue.cc:260] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.623468 8844 ts_tablet_manager.cc:1398] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:26.623934 6993 tablet_service.cc:1467] Processing CreateTablet for tablet be115ad80a4b42edbc6b2c630e54aedc (DEFAULT_TABLE table=rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.624047 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 5fe43b0765d9401a88c8e9a028ab3ef4 (DEFAULT_TABLE table=rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.624169 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be115ad80a4b42edbc6b2c630e54aedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.624218 8842 tablet_bootstrap.cc:492] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:26.624291 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fe43b0765d9401a88c8e9a028ab3ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.624300 8842 ts_tablet_manager.cc:1367] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:26.624747 8842 raft_consensus.cc:348] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.624850 8842 raft_consensus.cc:374] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.624804 7255 tablet_service.cc:1467] Processing CreateTablet for tablet be115ad80a4b42edbc6b2c630e54aedc (DEFAULT_TABLE table=rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.624881 8842 raft_consensus.cc:729] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.624994 8842 consensus_queue.cc:260] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.625254 8842 ts_tablet_manager.cc:1398] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:26.626173 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 5fe43b0765d9401a88c8e9a028ab3ef4 (DEFAULT_TABLE table=rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.626485 8843 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:26.626637 7123 tablet_service.cc:1467] Processing CreateTablet for tablet be115ad80a4b42edbc6b2c630e54aedc (DEFAULT_TABLE table=rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.626796 8841 tablet_bootstrap.cc:492] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:26.626876 8841 ts_tablet_manager.cc:1367] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240426 09:49:26.626896 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be115ad80a4b42edbc6b2c630e54aedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.627286 8841 raft_consensus.cc:348] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.627379 8841 raft_consensus.cc:374] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.627418 8841 raft_consensus.cc:729] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.627504 8843 tablet_bootstrap.cc:654] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.627549 8841 consensus_queue.cc:260] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.627840 8841 ts_tablet_manager.cc:1398] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:26.628041 8841 tablet_bootstrap.cc:492] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:26.628763 8841 tablet_bootstrap.cc:654] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.629549 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be115ad80a4b42edbc6b2c630e54aedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.629884 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 5fe43b0765d9401a88c8e9a028ab3ef4 (DEFAULT_TABLE table=rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:26.630123 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fe43b0765d9401a88c8e9a028ab3ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.631259 8844 tablet_bootstrap.cc:492] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:26.632016 8844 tablet_bootstrap.cc:654] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.632202 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fe43b0765d9401a88c8e9a028ab3ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:26.632303 8842 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:26.633150 8844 tablet_bootstrap.cc:492] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:26.633226 8844 ts_tablet_manager.cc:1367] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:26.633271 8842 tablet_bootstrap.cc:654] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.633638 8844 raft_consensus.cc:348] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.633728 8844 raft_consensus.cc:374] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.633774 8844 raft_consensus.cc:729] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.633891 8844 consensus_queue.cc:260] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.634163 8844 ts_tablet_manager.cc:1398] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:26.634191 8842 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:26.634258 8842 ts_tablet_manager.cc:1367] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:26.634317 8844 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:26.634567 8842 raft_consensus.cc:348] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:26.634665 8842 raft_consensus.cc:374] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.634694 8842 raft_consensus.cc:729] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.634830 8842 consensus_queue.cc:260] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:26.635087 8842 ts_tablet_manager.cc:1398] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:26.635315 8844 tablet_bootstrap.cc:654] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.635780 8841 tablet_bootstrap.cc:492] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:26.635859 8841 ts_tablet_manager.cc:1367] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240426 09:49:26.636283 8841 raft_consensus.cc:348] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.636379 8841 raft_consensus.cc:374] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.636406 8841 raft_consensus.cc:729] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.636518 8841 consensus_queue.cc:260] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.636818 8841 ts_tablet_manager.cc:1398] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:26.636914 8722 raft_consensus.cc:482] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:26.636996 8722 raft_consensus.cc:504] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.637343 8722 leader_election.cc:288] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:26.637727 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be115ad80a4b42edbc6b2c630e54aedc" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:26.637728 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be115ad80a4b42edbc6b2c630e54aedc" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:26.637835 7274 raft_consensus.cc:2454] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:26.638119 6948 leader_election.cc:302] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:26.638375 8606 raft_consensus.cc:2781] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:26.638438 8606 raft_consensus.cc:482] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:26.638475 8606 raft_consensus.cc:3037] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
W20240426 09:49:26.638383 6946 leader_election.cc:341] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:26.639463 8606 raft_consensus.cc:504] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.639701 8843 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:26.639777 8606 leader_election.cc:288] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:26.639803 8843 ts_tablet_manager.cc:1367] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:26.639946 8844 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:26.640018 8844 ts_tablet_manager.cc:1367] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:26.640103 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be115ad80a4b42edbc6b2c630e54aedc" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:26.640197 7274 raft_consensus.cc:3037] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.640223 8843 raft_consensus.cc:348] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:26.640345 8843 raft_consensus.cc:374] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.640378 8843 raft_consensus.cc:729] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.640414 8844 raft_consensus.cc:348] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:26.640497 8844 raft_consensus.cc:374] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.640530 8844 raft_consensus.cc:729] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.640501 8843 consensus_queue.cc:260] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:26.640631 8844 consensus_queue.cc:260] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:26.640792 8843 ts_tablet_manager.cc:1398] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:26.640882 8844 ts_tablet_manager.cc:1398] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:26.640954 8843 tablet_bootstrap.cc:492] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:26.641239 7274 raft_consensus.cc:2454] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:26.640038 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be115ad80a4b42edbc6b2c630e54aedc" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
W20240426 09:49:26.641672 6946 leader_election.cc:341] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:26.641974 8843 tablet_bootstrap.cc:654] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:26.642060 6948 leader_election.cc:302] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters: 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:26.642263 8606 raft_consensus.cc:2781] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:26.642346 8606 raft_consensus.cc:686] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:26.642478 8606 consensus_queue.cc:237] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.643833 8843 tablet_bootstrap.cc:492] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:26.643911 8843 ts_tablet_manager.cc:1367] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:26.644291 8843 raft_consensus.cc:348] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.644397 8843 raft_consensus.cc:374] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:26.644438 8843 raft_consensus.cc:729] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:26.644579 8843 consensus_queue.cc:260] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.644541 6734 catalog_manager.cc:5484] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:26.644819 8843 ts_tablet_manager.cc:1398] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:26.652045 8809 raft_consensus.cc:482] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:26.652165 8809 raft_consensus.cc:504] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.652572 8809 leader_election.cc:288] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:26.652915 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e130e7733534aedb221d7772a2d29ce" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:26.652977 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e130e7733534aedb221d7772a2d29ce" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:26.653053 7013 raft_consensus.cc:2454] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:26.653120 7274 raft_consensus.cc:2454] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:26.653337 6815 leader_election.cc:302] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:26.653541 8809 raft_consensus.cc:2781] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:26.653609 8809 raft_consensus.cc:482] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:26.653647 8809 raft_consensus.cc:3037] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.654479 8809 raft_consensus.cc:504] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.654881 8809 leader_election.cc:288] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:26.655138 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e130e7733534aedb221d7772a2d29ce" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:26.655162 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e130e7733534aedb221d7772a2d29ce" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:26.655254 7013 raft_consensus.cc:3037] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.655261 7274 raft_consensus.cc:3037] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.655984 7013 raft_consensus.cc:2454] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:26.656173 7274 raft_consensus.cc:2454] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:26.656306 6815 leader_election.cc:302] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:26.656515 8809 raft_consensus.cc:2781] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:26.656582 8809 raft_consensus.cc:686] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:26.656718 8809 consensus_queue.cc:237] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:26.658458 6740 catalog_manager.cc:5484] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:26.718133 8809 raft_consensus.cc:482] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:26.718274 8809 raft_consensus.cc:504] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:26.718653 8809 leader_election.cc:288] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:26.719034 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe43b0765d9401a88c8e9a028ab3ef4" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:26.719046 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe43b0765d9401a88c8e9a028ab3ef4" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:26.719154 7274 raft_consensus.cc:2454] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:26.719193 7142 raft_consensus.cc:2454] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:26.719441 6817 leader_election.cc:302] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:26.719620 8809 raft_consensus.cc:2781] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:26.719689 8809 raft_consensus.cc:482] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:26.719731 8809 raft_consensus.cc:3037] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.720567 8809 raft_consensus.cc:504] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:26.720881 8809 leader_election.cc:288] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:26.721226 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe43b0765d9401a88c8e9a028ab3ef4" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:26.721390 7142 raft_consensus.cc:3037] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.721263 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe43b0765d9401a88c8e9a028ab3ef4" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:26.721637 7274 raft_consensus.cc:3037] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.722489 7142 raft_consensus.cc:2454] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:26.722738 7274 raft_consensus.cc:2454] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:26.722805 6815 leader_election.cc:302] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:26.722976 8809 raft_consensus.cc:2781] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:26.723091 8809 raft_consensus.cc:686] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:26.723229 8809 consensus_queue.cc:237] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:26.725023 6740 catalog_manager.cc:5484] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
W20240426 09:49:26.840498 8905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:26.840741 8905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:26.844247 8905 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_15'
I20240426 09:49:26.937987 7142 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.939126 8809 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.939685 7275 raft_consensus.cc:1261] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.940263 8722 consensus_queue.cc:1035] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.943488 7013 raft_consensus.cc:1261] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.944135 7143 raft_consensus.cc:3037] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:26.945248 7143 raft_consensus.cc:1261] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.945875 7273 raft_consensus.cc:1261] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.946595 8764 consensus_queue.cc:1035] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.946975 8764 consensus_queue.cc:1035] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.949429 8722 consensus_queue.cc:1035] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:26.950409 7274 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:26.956476 8809 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 115.195 ms
time per row: 0.0575973 ms
I20240426 09:49:27.006750 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:27.012085 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 30206c7050b64ad2900c789f48d3390f (DEFAULT_TABLE table=rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.012401 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30206c7050b64ad2900c789f48d3390f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.013203 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 30206c7050b64ad2900c789f48d3390f (DEFAULT_TABLE table=rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.013325 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 6111d5f14ab5490692a69d193ae4aa45 (DEFAULT_TABLE table=rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.013468 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30206c7050b64ad2900c789f48d3390f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.013600 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6111d5f14ab5490692a69d193ae4aa45. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.013664 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 6111d5f14ab5490692a69d193ae4aa45 (DEFAULT_TABLE table=rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.013924 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6111d5f14ab5490692a69d193ae4aa45. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.014847 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 30206c7050b64ad2900c789f48d3390f (DEFAULT_TABLE table=rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.015066 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30206c7050b64ad2900c789f48d3390f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.015120 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 6111d5f14ab5490692a69d193ae4aa45 (DEFAULT_TABLE table=rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.015349 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6111d5f14ab5490692a69d193ae4aa45. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.016554 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 154f16534c294c99b1fcb1735b838d66 (DEFAULT_TABLE table=rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.016795 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 154f16534c294c99b1fcb1735b838d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.017037 8844 tablet_bootstrap.cc:492] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:27.017252 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 154f16534c294c99b1fcb1735b838d66 (DEFAULT_TABLE table=rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.017453 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 154f16534c294c99b1fcb1735b838d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.017910 8844 tablet_bootstrap.cc:654] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.018558 8841 tablet_bootstrap.cc:492] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:27.019115 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 154f16534c294c99b1fcb1735b838d66 (DEFAULT_TABLE table=rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.019344 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 154f16534c294c99b1fcb1735b838d66. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.019389 8841 tablet_bootstrap.cc:654] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.020013 8842 tablet_bootstrap.cc:492] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:27.020515 8841 tablet_bootstrap.cc:492] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:27.020589 8841 ts_tablet_manager.cc:1367] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:27.020730 8842 tablet_bootstrap.cc:654] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.021077 8841 raft_consensus.cc:348] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.021189 8841 raft_consensus.cc:374] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.021222 8841 raft_consensus.cc:729] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.021343 8841 consensus_queue.cc:260] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.021538 8844 tablet_bootstrap.cc:492] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:27.021627 8844 ts_tablet_manager.cc:1367] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:49:27.021670 8841 ts_tablet_manager.cc:1398] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.021858 8841 tablet_bootstrap.cc:492] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:27.022118 8844 raft_consensus.cc:348] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.022194 8844 raft_consensus.cc:374] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.022214 8844 raft_consensus.cc:729] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.022300 8844 consensus_queue.cc:260] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.022552 8844 ts_tablet_manager.cc:1398] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.022667 8844 tablet_bootstrap.cc:492] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:27.022840 8843 tablet_bootstrap.cc:492] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:27.023483 8844 tablet_bootstrap.cc:654] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.023732 8843 tablet_bootstrap.cc:654] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.023950 8841 tablet_bootstrap.cc:654] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.024942 8841 tablet_bootstrap.cc:492] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:27.025017 8841 ts_tablet_manager.cc:1367] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:27.025430 8841 raft_consensus.cc:348] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:27.025522 8841 raft_consensus.cc:374] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.025548 8841 raft_consensus.cc:729] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.025660 8841 consensus_queue.cc:260] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:27.025918 8841 ts_tablet_manager.cc:1398] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.026527 8843 tablet_bootstrap.cc:492] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:27.026609 8843 ts_tablet_manager.cc:1367] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:27.027065 8843 raft_consensus.cc:348] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.027166 8843 raft_consensus.cc:374] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.027202 8843 raft_consensus.cc:729] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.027218 8844 tablet_bootstrap.cc:492] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:27.027288 8844 ts_tablet_manager.cc:1367] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:49:27.027333 8843 consensus_queue.cc:260] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.027544 8842 tablet_bootstrap.cc:492] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:27.027602 8843 ts_tablet_manager.cc:1398] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.027633 8842 ts_tablet_manager.cc:1367] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240426 09:49:27.027673 8844 raft_consensus.cc:348] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.027765 8843 tablet_bootstrap.cc:492] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:27.027772 8844 raft_consensus.cc:374] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.027805 8844 raft_consensus.cc:729] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.027910 8844 consensus_queue.cc:260] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.028033 8842 raft_consensus.cc:348] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:27.028142 8842 raft_consensus.cc:374] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.028175 8842 raft_consensus.cc:729] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.028187 8844 ts_tablet_manager.cc:1398] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:27.028277 8842 consensus_queue.cc:260] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:27.028580 8843 tablet_bootstrap.cc:654] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.028577 8842 ts_tablet_manager.cc:1398] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:27.028734 8842 tablet_bootstrap.cc:492] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:27.029928 8842 tablet_bootstrap.cc:654] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.029969 8606 raft_consensus.cc:482] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:27.030077 8843 tablet_bootstrap.cc:492] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:27.030048 8606 raft_consensus.cc:504] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:27.030140 8843 ts_tablet_manager.cc:1367] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:27.030401 8606 leader_election.cc:288] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:27.030560 8843 raft_consensus.cc:348] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:27.030649 8843 raft_consensus.cc:374] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.030679 8843 raft_consensus.cc:729] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.030803 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6111d5f14ab5490692a69d193ae4aa45" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:27.030831 8843 consensus_queue.cc:260] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:27.030903 6880 raft_consensus.cc:2454] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:27.031121 8843 ts_tablet_manager.cc:1398] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:27.030908 8842 tablet_bootstrap.cc:492] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:27.031215 8842 ts_tablet_manager.cc:1367] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:27.031231 6948 leader_election.cc:302] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:27.031360 8606 raft_consensus.cc:2781] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:27.031409 8606 raft_consensus.cc:482] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:27.031445 8606 raft_consensus.cc:3037] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.031632 8842 raft_consensus.cc:348] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.031719 8842 raft_consensus.cc:374] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.031749 8842 raft_consensus.cc:729] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.031890 8842 consensus_queue.cc:260] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.032375 8842 ts_tablet_manager.cc:1398] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.032527 8842 tablet_bootstrap.cc:492] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:27.032485 8606 raft_consensus.cc:504] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:27.032639 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6111d5f14ab5490692a69d193ae4aa45" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:27.032747 7143 raft_consensus.cc:2454] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:27.032824 8606 leader_election.cc:288] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:27.033486 8842 tablet_bootstrap.cc:654] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.033685 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6111d5f14ab5490692a69d193ae4aa45" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:27.033795 6880 raft_consensus.cc:3037] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.034375 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6111d5f14ab5490692a69d193ae4aa45" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:27.034466 7143 raft_consensus.cc:3037] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.034713 6880 raft_consensus.cc:2454] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:27.035140 6948 leader_election.cc:302] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:27.036417 7143 raft_consensus.cc:2454] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:27.036667 8606 raft_consensus.cc:2781] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:27.036743 8606 raft_consensus.cc:686] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:27.036868 8606 consensus_queue.cc:237] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:27.038863 6740 catalog_manager.cc:5484] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:27.040323 8842 tablet_bootstrap.cc:492] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:27.040411 8842 ts_tablet_manager.cc:1367] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:49:27.040933 8842 raft_consensus.cc:348] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.041062 8842 raft_consensus.cc:374] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.041088 8842 raft_consensus.cc:729] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.041188 8842 consensus_queue.cc:260] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.041493 8842 ts_tablet_manager.cc:1398] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.047403 8722 raft_consensus.cc:482] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:27.047498 8722 raft_consensus.cc:504] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.047819 8722 leader_election.cc:288] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:27.048152 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30206c7050b64ad2900c789f48d3390f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:27.048188 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30206c7050b64ad2900c789f48d3390f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:27.048281 7274 raft_consensus.cc:2454] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:27.048296 6880 raft_consensus.cc:2454] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:27.048537 6948 leader_election.cc:302] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:27.048688 8722 raft_consensus.cc:2781] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:27.048743 8722 raft_consensus.cc:482] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:27.048792 8722 raft_consensus.cc:3037] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.049907 8722 raft_consensus.cc:504] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.050202 8722 leader_election.cc:288] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:27.050525 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30206c7050b64ad2900c789f48d3390f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:27.050552 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30206c7050b64ad2900c789f48d3390f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:27.050634 7274 raft_consensus.cc:3037] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.050650 6880 raft_consensus.cc:3037] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.051409 7274 raft_consensus.cc:2454] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:27.051420 6880 raft_consensus.cc:2454] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:27.051784 6948 leader_election.cc:302] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:27.051957 8722 raft_consensus.cc:2781] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:27.052052 8722 raft_consensus.cc:686] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:27.052194 8722 consensus_queue.cc:237] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.054025 6740 catalog_manager.cc:5484] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:27.151664 8584 raft_consensus.cc:482] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:27.151782 8584 raft_consensus.cc:504] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.152159 8584 leader_election.cc:288] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:27.152496 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "154f16534c294c99b1fcb1735b838d66" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:27.152530 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "154f16534c294c99b1fcb1735b838d66" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:27.152632 7142 raft_consensus.cc:2454] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:27.152648 6880 raft_consensus.cc:2454] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:27.152947 7210 leader_election.cc:302] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:27.153155 8584 raft_consensus.cc:2781] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:27.153216 8584 raft_consensus.cc:482] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:27.153264 8584 raft_consensus.cc:3037] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.154099 8584 raft_consensus.cc:504] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.154412 8584 leader_election.cc:288] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:27.154825 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "154f16534c294c99b1fcb1735b838d66" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:27.154834 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "154f16534c294c99b1fcb1735b838d66" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:27.154937 7142 raft_consensus.cc:3037] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.154937 6880 raft_consensus.cc:3037] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.155678 6880 raft_consensus.cc:2454] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:27.155864 7142 raft_consensus.cc:2454] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:27.155952 7210 leader_election.cc:302] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:27.156234 8584 raft_consensus.cc:2781] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:27.156318 8584 raft_consensus.cc:686] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:27.156467 8584 consensus_queue.cc:237] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.158428 6740 catalog_manager.cc:5484] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
W20240426 09:49:27.287103 8959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:27.287348 8959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:27.290181 8959 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_16'
I20240426 09:49:27.370240 7142 raft_consensus.cc:1261] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.370633 8722 consensus_queue.cc:1035] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.370954 7275 raft_consensus.cc:1261] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.371043 6882 raft_consensus.cc:1261] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.371356 6881 raft_consensus.cc:1261] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.371547 8722 consensus_queue.cc:1035] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.371835 8606 consensus_queue.cc:1035] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.371932 8722 consensus_queue.cc:1035] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.384352 8830 mvcc.cc:205] Tried to move back new op lower bound from 7021055866342608896 to 7021055864984334336. Current Snapshot: MvccSnapshot[applied={T|T < 7021055866342608896}]
I20240426 09:49:27.399338 8891 mvcc.cc:205] Tried to move back new op lower bound from 7021055866342608896 to 7021055864984334336. Current Snapshot: MvccSnapshot[applied={T|T < 7021055866342608896}]
I20240426 09:49:27.416244 7142 raft_consensus.cc:1261] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.416339 6882 raft_consensus.cc:1261] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.416692 8705 consensus_queue.cc:1035] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.417068 8584 consensus_queue.cc:1035] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 115.158 ms
time per row: 0.0575789 ms
I20240426 09:49:27.447559 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:27.453652 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 0f67d94463ce46708246808da0b100e9 (DEFAULT_TABLE table=rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.453987 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f67d94463ce46708246808da0b100e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.454182 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 0f67d94463ce46708246808da0b100e9 (DEFAULT_TABLE table=rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.454452 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f67d94463ce46708246808da0b100e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.454514 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 4189112964fb425aac074352dd873415 (DEFAULT_TABLE table=rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.454865 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4189112964fb425aac074352dd873415. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.456627 8843 tablet_bootstrap.cc:492] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:27.456591 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9c1e00452f4f32b31e8b336892df49 (DEFAULT_TABLE table=rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.456714 8844 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:27.457044 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9c1e00452f4f32b31e8b336892df49 (DEFAULT_TABLE table=rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.457268 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9c1e00452f4f32b31e8b336892df49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.457446 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9c1e00452f4f32b31e8b336892df49 (DEFAULT_TABLE table=rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.457561 8843 tablet_bootstrap.cc:654] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.457561 8844 tablet_bootstrap.cc:654] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.457659 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9c1e00452f4f32b31e8b336892df49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.454514 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 0f67d94463ce46708246808da0b100e9 (DEFAULT_TABLE table=rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.458292 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 4189112964fb425aac074352dd873415 (DEFAULT_TABLE table=rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.458516 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4189112964fb425aac074352dd873415. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.459268 8841 tablet_bootstrap.cc:492] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:27.460176 8841 tablet_bootstrap.cc:654] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.460281 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 4189112964fb425aac074352dd873415 (DEFAULT_TABLE table=rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.460393 8843 tablet_bootstrap.cc:492] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:27.460469 8843 ts_tablet_manager.cc:1367] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:49:27.460582 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4189112964fb425aac074352dd873415. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.460970 8843 raft_consensus.cc:348] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.461112 8843 raft_consensus.cc:374] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.461151 8843 raft_consensus.cc:729] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.461280 8843 consensus_queue.cc:260] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.461565 8843 ts_tablet_manager.cc:1398] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:27.461719 8841 tablet_bootstrap.cc:492] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:27.461812 8841 ts_tablet_manager.cc:1367] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:27.462287 8841 raft_consensus.cc:348] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.462409 8841 raft_consensus.cc:374] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.462450 8841 raft_consensus.cc:729] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.462560 8841 consensus_queue.cc:260] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.462637 8842 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:27.462918 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9c1e00452f4f32b31e8b336892df49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.463186 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f67d94463ce46708246808da0b100e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.463621 8842 tablet_bootstrap.cc:654] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.464633 8841 ts_tablet_manager.cc:1398] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:27.465421 8841 tablet_bootstrap.cc:492] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:27.466071 8843 tablet_bootstrap.cc:492] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:27.466184 8844 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:27.466254 8844 ts_tablet_manager.cc:1367] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:27.466696 8844 raft_consensus.cc:348] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.466828 8844 raft_consensus.cc:374] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.466863 8844 raft_consensus.cc:729] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.466979 8844 consensus_queue.cc:260] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.467162 8841 tablet_bootstrap.cc:654] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.467253 8844 ts_tablet_manager.cc:1398] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.467399 8844 tablet_bootstrap.cc:492] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:27.467690 8843 tablet_bootstrap.cc:654] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.468266 8841 tablet_bootstrap.cc:492] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:27.468338 8841 ts_tablet_manager.cc:1367] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:27.468348 8844 tablet_bootstrap.cc:654] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.468689 8841 raft_consensus.cc:348] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.468775 8841 raft_consensus.cc:374] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.468797 8841 raft_consensus.cc:729] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.468892 8841 consensus_queue.cc:260] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.469025 8842 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:27.469113 8842 ts_tablet_manager.cc:1367] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:27.469180 8841 ts_tablet_manager.cc:1398] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:27.469338 8841 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:27.469528 8842 raft_consensus.cc:348] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.469630 8842 raft_consensus.cc:374] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.469668 8842 raft_consensus.cc:729] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.469806 8842 consensus_queue.cc:260] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.470084 8842 ts_tablet_manager.cc:1398] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.470211 8841 tablet_bootstrap.cc:654] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.471223 8844 tablet_bootstrap.cc:492] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:27.471299 8844 ts_tablet_manager.cc:1367] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:27.471545 8841 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:27.471695 8844 raft_consensus.cc:348] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.471815 8844 raft_consensus.cc:374] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.471853 8844 raft_consensus.cc:729] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.472095 8844 consensus_queue.cc:260] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.472383 8844 ts_tablet_manager.cc:1398] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.472537 8844 tablet_bootstrap.cc:492] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:27.473527 8844 tablet_bootstrap.cc:654] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:27.473527 8843 tablet_bootstrap.cc:492] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:27.473623 8843 ts_tablet_manager.cc:1367] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:27.473917 8841 ts_tablet_manager.cc:1367] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:27.474100 8843 raft_consensus.cc:348] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.474208 8843 raft_consensus.cc:374] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.474246 8843 raft_consensus.cc:729] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.474381 8843 consensus_queue.cc:260] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.474653 8843 ts_tablet_manager.cc:1398] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.474622 8841 raft_consensus.cc:348] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.474818 8841 raft_consensus.cc:374] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.474915 8841 raft_consensus.cc:729] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.475087 8841 consensus_queue.cc:260] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.475378 8841 ts_tablet_manager.cc:1398] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:27.476119 8844 tablet_bootstrap.cc:492] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:27.476189 8844 ts_tablet_manager.cc:1367] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:27.476568 8844 raft_consensus.cc:348] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.476661 8844 raft_consensus.cc:374] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:27.476689 8844 raft_consensus.cc:729] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:27.476784 8844 consensus_queue.cc:260] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.477037 8844 ts_tablet_manager.cc:1398] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:27.478089 8705 raft_consensus.cc:482] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:27.478175 8705 raft_consensus.cc:504] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.478327 8826 raft_consensus.cc:482] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:27.478401 8826 raft_consensus.cc:504] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.478550 8705 leader_election.cc:288] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:27.479022 8826 leader_election.cc:288] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:27.479007 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4189112964fb425aac074352dd873415" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:27.479032 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4189112964fb425aac074352dd873415" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:27.479136 6882 raft_consensus.cc:2454] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:27.479146 7013 raft_consensus.cc:2454] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:27.479462 7210 leader_election.cc:302] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:27.479594 8705 raft_consensus.cc:2781] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:27.479637 8705 raft_consensus.cc:482] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:27.479673 8705 raft_consensus.cc:3037] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.479674 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f67d94463ce46708246808da0b100e9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:27.479760 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f67d94463ce46708246808da0b100e9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:27.479816 7275 raft_consensus.cc:2454] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:27.479882 7011 raft_consensus.cc:2454] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:27.480145 7077 leader_election.cc:302] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:27.480782 8705 raft_consensus.cc:504] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.481072 8705 leader_election.cc:288] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:27.481199 8826 raft_consensus.cc:2781] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:27.481252 8826 raft_consensus.cc:482] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:27.481289 8826 raft_consensus.cc:3037] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.481389 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4189112964fb425aac074352dd873415" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:27.481405 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4189112964fb425aac074352dd873415" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:27.481508 6882 raft_consensus.cc:3037] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.481509 7011 raft_consensus.cc:3037] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.482261 7011 raft_consensus.cc:2454] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:27.482262 8826 raft_consensus.cc:504] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.482465 6882 raft_consensus.cc:2454] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:27.482538 7208 leader_election.cc:302] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:27.482607 8826 leader_election.cc:288] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:27.483246 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f67d94463ce46708246808da0b100e9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:27.483259 8705 raft_consensus.cc:2781] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:27.483342 7011 raft_consensus.cc:3037] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.483354 8705 raft_consensus.cc:686] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:27.483512 8705 consensus_queue.cc:237] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:27.484205 7011 raft_consensus.cc:2454] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:27.484239 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f67d94463ce46708246808da0b100e9" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:27.484328 7275 raft_consensus.cc:3037] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.484556 7077 leader_election.cc:302] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:27.484715 8826 raft_consensus.cc:2781] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:27.484818 8826 raft_consensus.cc:686] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:27.484942 8826 consensus_queue.cc:237] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.485220 7275 raft_consensus.cc:2454] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:27.486106 6736 catalog_manager.cc:5484] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:27.487810 6734 catalog_manager.cc:5484] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:27.654357 8705 raft_consensus.cc:482] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:27.654493 8705 raft_consensus.cc:504] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.654935 8705 leader_election.cc:288] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:27.655390 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9c1e00452f4f32b31e8b336892df49" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:27.655442 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9c1e00452f4f32b31e8b336892df49" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:27.655525 7011 raft_consensus.cc:2454] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:27.655622 7142 raft_consensus.cc:2454] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:27.655839 7208 leader_election.cc:302] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:27.656002 8705 raft_consensus.cc:2781] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:27.656085 8705 raft_consensus.cc:482] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:27.656138 8705 raft_consensus.cc:3037] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.656950 8705 raft_consensus.cc:504] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.657272 8705 leader_election.cc:288] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:27.657666 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9c1e00452f4f32b31e8b336892df49" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:27.657655 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9c1e00452f4f32b31e8b336892df49" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:27.657785 7142 raft_consensus.cc:3037] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.657799 7011 raft_consensus.cc:3037] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:27.658805 7142 raft_consensus.cc:2454] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:27.658805 7011 raft_consensus.cc:2454] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:27.659104 7208 leader_election.cc:302] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:27.659271 8705 raft_consensus.cc:2781] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:27.659382 8705 raft_consensus.cc:686] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:27.659523 8705 consensus_queue.cc:237] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:27.661403 6734 catalog_manager.cc:5484] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
W20240426 09:49:27.812731 9007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:27.813158 9007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:27.816350 9007 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_17'
I20240426 09:49:27.910352 7011 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.910833 8584 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.911022 6880 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.911408 8584 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.912410 7142 raft_consensus.cc:1261] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.913102 7013 raft_consensus.cc:1261] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.913611 8705 consensus_queue.cc:1035] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.913918 8705 consensus_queue.cc:1035] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.924752 7275 raft_consensus.cc:1261] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.925930 8826 consensus_queue.cc:1035] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.929100 7013 raft_consensus.cc:1261] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:27.929332 8818 mvcc.cc:205] Tried to move back new op lower bound from 7021055868553232384 to 7021055867534794752. Current Snapshot: MvccSnapshot[applied={T|T < 7021055868553232384}]
I20240426 09:49:27.934203 8826 consensus_queue.cc:1035] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:27.952973 9041 mvcc.cc:205] Tried to move back new op lower bound from 7021055868553232384 to 7021055867534794752. Current Snapshot: MvccSnapshot[applied={T|T < 7021055868553232384}]
INSERT report
rows total: 2000
time total: 134.758 ms
time per row: 0.0673789 ms
I20240426 09:49:27.993002 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:27.998817 6860 tablet_service.cc:1467] Processing CreateTablet for tablet c31c4fd362e84114b8dc45faa0ee8383 (DEFAULT_TABLE table=rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.998870 7122 tablet_service.cc:1467] Processing CreateTablet for tablet c31c4fd362e84114b8dc45faa0ee8383 (DEFAULT_TABLE table=rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.998922 6993 tablet_service.cc:1467] Processing CreateTablet for tablet c31c4fd362e84114b8dc45faa0ee8383 (DEFAULT_TABLE table=rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:27.999131 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31c4fd362e84114b8dc45faa0ee8383. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.999136 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31c4fd362e84114b8dc45faa0ee8383. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:27.999229 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c31c4fd362e84114b8dc45faa0ee8383. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.001211 9057 tablet_bootstrap.cc:492] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:28.001863 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2fd1740d544f5d98741aecb64cdfc7 (DEFAULT_TABLE table=rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.001900 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2fd1740d544f5d98741aecb64cdfc7 (DEFAULT_TABLE table=rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.001905 6861 tablet_service.cc:1467] Processing CreateTablet for tablet c8ab2d56585a450a8b2d7d9497f7c9cd (DEFAULT_TABLE table=rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.001905 7123 tablet_service.cc:1467] Processing CreateTablet for tablet c8ab2d56585a450a8b2d7d9497f7c9cd (DEFAULT_TABLE table=rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.002115 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2fd1740d544f5d98741aecb64cdfc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.002116 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2fd1740d544f5d98741aecb64cdfc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.002153 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ab2d56585a450a8b2d7d9497f7c9cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.002242 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ab2d56585a450a8b2d7d9497f7c9cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.004263 9059 tablet_bootstrap.cc:492] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:28.004519 9057 tablet_bootstrap.cc:654] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.004519 6993 tablet_service.cc:1467] Processing CreateTablet for tablet c8ab2d56585a450a8b2d7d9497f7c9cd (DEFAULT_TABLE table=rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.004953 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ab2d56585a450a8b2d7d9497f7c9cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.005131 9059 tablet_bootstrap.cc:654] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.005604 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 1c2fd1740d544f5d98741aecb64cdfc7 (DEFAULT_TABLE table=rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.005779 9058 tablet_bootstrap.cc:492] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:28.005808 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2fd1740d544f5d98741aecb64cdfc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.006690 9058 tablet_bootstrap.cc:654] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.007522 9057 tablet_bootstrap.cc:492] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:28.007614 9057 ts_tablet_manager.cc:1367] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:28.007879 9060 tablet_bootstrap.cc:492] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:28.008118 9057 raft_consensus.cc:348] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.008230 9057 raft_consensus.cc:374] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.008261 9057 raft_consensus.cc:729] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.008407 9057 consensus_queue.cc:260] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.008669 9060 tablet_bootstrap.cc:654] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.008710 9057 ts_tablet_manager.cc:1398] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:28.009027 9057 tablet_bootstrap.cc:492] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:28.009766 9060 tablet_bootstrap.cc:492] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:28.009768 9059 tablet_bootstrap.cc:492] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:28.009851 9059 ts_tablet_manager.cc:1367] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:28.009914 9060 ts_tablet_manager.cc:1367] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:28.009982 9057 tablet_bootstrap.cc:654] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.010326 9059 raft_consensus.cc:348] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.010505 9059 raft_consensus.cc:374] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.010486 9060 raft_consensus.cc:348] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.010593 9059 raft_consensus.cc:729] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.010656 9060 raft_consensus.cc:374] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.010756 9060 raft_consensus.cc:729] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.010830 9059 consensus_queue.cc:260] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.010950 9060 consensus_queue.cc:260] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.011054 9058 tablet_bootstrap.cc:492] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:28.011153 9058 ts_tablet_manager.cc:1367] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240426 09:49:28.011384 9060 ts_tablet_manager.cc:1398] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:28.011916 9059 ts_tablet_manager.cc:1398] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:28.012132 9059 tablet_bootstrap.cc:492] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:28.012316 9058 raft_consensus.cc:348] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.012425 9058 raft_consensus.cc:374] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.012459 9058 raft_consensus.cc:729] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.012621 9058 consensus_queue.cc:260] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.013026 9058 ts_tablet_manager.cc:1398] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:28.013120 9059 tablet_bootstrap.cc:654] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.013360 9058 tablet_bootstrap.cc:492] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:28.014864 9058 tablet_bootstrap.cc:654] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.015647 9057 tablet_bootstrap.cc:492] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:28.015744 9057 ts_tablet_manager.cc:1367] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:28.016198 9057 raft_consensus.cc:348] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.016310 9057 raft_consensus.cc:374] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.016353 9057 raft_consensus.cc:729] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.016511 9059 tablet_bootstrap.cc:492] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:28.016485 9057 consensus_queue.cc:260] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.016697 9059 ts_tablet_manager.cc:1367] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:28.016798 9057 ts_tablet_manager.cc:1398] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:28.017148 9059 raft_consensus.cc:348] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.017364 9059 raft_consensus.cc:374] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.017406 9059 raft_consensus.cc:729] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.017525 9059 consensus_queue.cc:260] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.018443 9059 ts_tablet_manager.cc:1398] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:28.018642 9059 tablet_bootstrap.cc:492] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:28.019100 9058 tablet_bootstrap.cc:492] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:28.019189 9058 ts_tablet_manager.cc:1367] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240426 09:49:28.019536 9058 raft_consensus.cc:348] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.019624 9058 raft_consensus.cc:374] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.019624 9059 tablet_bootstrap.cc:654] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.019658 9058 raft_consensus.cc:729] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.019804 9058 consensus_queue.cc:260] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.020300 9058 ts_tablet_manager.cc:1398] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:28.020531 9058 tablet_bootstrap.cc:492] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:28.021076 9059 tablet_bootstrap.cc:492] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:28.021214 9059 ts_tablet_manager.cc:1367] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:28.021486 9058 tablet_bootstrap.cc:654] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.021667 9059 raft_consensus.cc:348] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.021850 9059 raft_consensus.cc:374] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.021936 9059 raft_consensus.cc:729] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.022162 9059 consensus_queue.cc:260] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.022502 9059 ts_tablet_manager.cc:1398] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:28.024384 9058 tablet_bootstrap.cc:492] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:28.024468 9058 ts_tablet_manager.cc:1367] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:49:28.024951 9058 raft_consensus.cc:348] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.025038 9058 raft_consensus.cc:374] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.025074 9058 raft_consensus.cc:729] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.025200 9058 consensus_queue.cc:260] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.025481 9058 ts_tablet_manager.cc:1398] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:28.039891 8892 raft_consensus.cc:482] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:28.039997 8892 raft_consensus.cc:504] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.040359 8892 leader_election.cc:288] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:28.040753 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ab2d56585a450a8b2d7d9497f7c9cd" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:28.040733 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ab2d56585a450a8b2d7d9497f7c9cd" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:28.040894 7142 raft_consensus.cc:2454] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:28.040894 7011 raft_consensus.cc:2454] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:28.041179 6815 leader_election.cc:302] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:28.041348 8892 raft_consensus.cc:2781] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:28.041417 8892 raft_consensus.cc:482] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:28.041467 8892 raft_consensus.cc:3037] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.042382 8892 raft_consensus.cc:504] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.042724 8892 leader_election.cc:288] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:28.043053 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ab2d56585a450a8b2d7d9497f7c9cd" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:28.043053 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ab2d56585a450a8b2d7d9497f7c9cd" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:28.043161 7011 raft_consensus.cc:3037] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.043161 7142 raft_consensus.cc:3037] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.044100 7142 raft_consensus.cc:2454] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:28.044100 7011 raft_consensus.cc:2454] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:28.044385 6815 leader_election.cc:302] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:28.044565 9070 raft_consensus.cc:2781] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:28.044637 9070 raft_consensus.cc:686] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:28.044796 9070 consensus_queue.cc:237] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.046722 6734 catalog_manager.cc:5484] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:28.114486 8988 raft_consensus.cc:482] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:28.114596 8988 raft_consensus.cc:504] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.114964 8988 leader_election.cc:288] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:28.115290 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31c4fd362e84114b8dc45faa0ee8383" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:28.115290 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31c4fd362e84114b8dc45faa0ee8383" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:28.115425 6882 raft_consensus.cc:2454] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:28.115425 7142 raft_consensus.cc:2454] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:28.115689 6946 leader_election.cc:302] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:28.115931 8988 raft_consensus.cc:2781] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:28.115988 8988 raft_consensus.cc:482] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:28.116014 8988 raft_consensus.cc:3037] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.116778 8988 raft_consensus.cc:504] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.117069 8988 leader_election.cc:288] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:28.117399 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31c4fd362e84114b8dc45faa0ee8383" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:28.117405 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c31c4fd362e84114b8dc45faa0ee8383" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:28.117514 6882 raft_consensus.cc:3037] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.117514 7142 raft_consensus.cc:3037] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.118439 6882 raft_consensus.cc:2454] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:28.118439 7142 raft_consensus.cc:2454] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:28.118733 6948 leader_election.cc:302] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:28.118929 8988 raft_consensus.cc:2781] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:28.119099 8988 raft_consensus.cc:686] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:28.119247 8988 consensus_queue.cc:237] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.121142 6734 catalog_manager.cc:5484] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:28.235926 9070 raft_consensus.cc:482] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:28.236061 9070 raft_consensus.cc:504] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.236454 9070 leader_election.cc:288] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:28.236807 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2fd1740d544f5d98741aecb64cdfc7" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:28.236835 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2fd1740d544f5d98741aecb64cdfc7" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:28.236927 7142 raft_consensus.cc:2454] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:28.236980 7275 raft_consensus.cc:2454] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:28.237202 6815 leader_election.cc:302] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:28.237372 9070 raft_consensus.cc:2781] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:28.237419 9070 raft_consensus.cc:482] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:28.237447 9070 raft_consensus.cc:3037] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.238303 9070 raft_consensus.cc:504] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.238606 9070 leader_election.cc:288] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:28.238937 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2fd1740d544f5d98741aecb64cdfc7" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:28.239037 7142 raft_consensus.cc:3037] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.239012 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c2fd1740d544f5d98741aecb64cdfc7" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:28.239130 7275 raft_consensus.cc:3037] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.239935 7142 raft_consensus.cc:2454] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:28.240134 7275 raft_consensus.cc:2454] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:28.240386 6817 leader_election.cc:302] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:28.240573 9070 raft_consensus.cc:2781] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:28.240644 9070 raft_consensus.cc:686] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:28.240832 9070 consensus_queue.cc:237] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.244061 6734 catalog_manager.cc:5484] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
W20240426 09:49:28.411991 9075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:28.412231 9075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:28.415385 9075 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_18'
I20240426 09:49:28.442968 9070 consensus_queue.cc:1035] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:28.446288 8892 consensus_queue.cc:1035] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:28.503912 7142 raft_consensus.cc:1261] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:28.504384 9070 consensus_queue.cc:1035] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:28.506698 7143 raft_consensus.cc:1261] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:28.507086 9072 consensus_queue.cc:1035] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:28.507189 7275 raft_consensus.cc:1261] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:28.507866 8892 consensus_queue.cc:1035] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:28.508867 6882 raft_consensus.cc:1261] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:28.510279 8988 consensus_queue.cc:1035] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 117.308 ms
time per row: 0.0586538 ms
I20240426 09:49:28.578971 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalance_test_table_19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:28.584338 7255 tablet_service.cc:1467] Processing CreateTablet for tablet a42302cab4524e7a85defe4da40be268 (DEFAULT_TABLE table=rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.584328 6993 tablet_service.cc:1467] Processing CreateTablet for tablet a42302cab4524e7a85defe4da40be268 (DEFAULT_TABLE table=rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.584462 6860 tablet_service.cc:1467] Processing CreateTablet for tablet a42302cab4524e7a85defe4da40be268 (DEFAULT_TABLE table=rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.584638 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42302cab4524e7a85defe4da40be268. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.584643 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42302cab4524e7a85defe4da40be268. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.584692 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42302cab4524e7a85defe4da40be268. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.586477 9129 tablet_bootstrap.cc:492] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:28.587311 7123 tablet_service.cc:1467] Processing CreateTablet for tablet cd377032e7cd4b4998424f8685ad5b85 (DEFAULT_TABLE table=rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.587472 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 295e6e95d86c47c88a09e47448c93d2e (DEFAULT_TABLE table=rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.587571 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd377032e7cd4b4998424f8685ad5b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.587718 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 295e6e95d86c47c88a09e47448c93d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.587939 9129 tablet_bootstrap.cc:654] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.590099 7255 tablet_service.cc:1467] Processing CreateTablet for tablet cd377032e7cd4b4998424f8685ad5b85 (DEFAULT_TABLE table=rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.590157 6993 tablet_service.cc:1467] Processing CreateTablet for tablet cd377032e7cd4b4998424f8685ad5b85 (DEFAULT_TABLE table=rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.590231 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 295e6e95d86c47c88a09e47448c93d2e (DEFAULT_TABLE table=rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.590263 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 295e6e95d86c47c88a09e47448c93d2e (DEFAULT_TABLE table=rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:28.590322 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd377032e7cd4b4998424f8685ad5b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.590373 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd377032e7cd4b4998424f8685ad5b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.590440 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 295e6e95d86c47c88a09e47448c93d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.590463 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 295e6e95d86c47c88a09e47448c93d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:28.592660 9131 tablet_bootstrap.cc:492] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:28.593595 9130 tablet_bootstrap.cc:492] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:28.593775 9133 tablet_bootstrap.cc:492] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:28.593902 9131 tablet_bootstrap.cc:654] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.594439 9130 tablet_bootstrap.cc:654] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.594604 9133 tablet_bootstrap.cc:654] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.594935 9129 tablet_bootstrap.cc:492] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:28.595077 9129 ts_tablet_manager.cc:1367] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:28.595762 9129 raft_consensus.cc:348] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:28.596020 9131 tablet_bootstrap.cc:492] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:28.596052 9129 raft_consensus.cc:374] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.596099 9129 raft_consensus.cc:729] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.596103 9131 ts_tablet_manager.cc:1367] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:28.596287 9129 consensus_queue.cc:260] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:28.596606 9129 ts_tablet_manager.cc:1398] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:28.596478 9131 raft_consensus.cc:348] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:28.596801 9131 raft_consensus.cc:374] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.596850 9131 raft_consensus.cc:729] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.597007 9131 consensus_queue.cc:260] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:28.597443 9133 tablet_bootstrap.cc:492] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:28.597443 9129 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:28.597570 9133 ts_tablet_manager.cc:1367] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:28.597923 9131 ts_tablet_manager.cc:1398] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:28.598110 9131 tablet_bootstrap.cc:492] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:28.598111 9133 raft_consensus.cc:348] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.598294 9133 raft_consensus.cc:374] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.598366 9133 raft_consensus.cc:729] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.598656 9129 tablet_bootstrap.cc:654] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.598686 9133 consensus_queue.cc:260] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.599098 9131 tablet_bootstrap.cc:654] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.599120 9133 ts_tablet_manager.cc:1398] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:28.599300 9133 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:28.600243 9133 tablet_bootstrap.cc:654] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.600286 9131 tablet_bootstrap.cc:492] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:28.600375 9131 ts_tablet_manager.cc:1367] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:28.600772 9131 raft_consensus.cc:348] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.600879 9131 raft_consensus.cc:374] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.600926 9131 raft_consensus.cc:729] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.600960 9130 tablet_bootstrap.cc:492] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:28.601076 9130 ts_tablet_manager.cc:1367] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:28.601054 9131 consensus_queue.cc:260] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.601349 9131 ts_tablet_manager.cc:1398] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:28.601505 9131 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:28.601644 9133 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:28.601730 9133 ts_tablet_manager.cc:1367] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:28.601962 9130 raft_consensus.cc:348] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:28.602061 9130 raft_consensus.cc:374] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.602092 9130 raft_consensus.cc:729] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.602201 9133 raft_consensus.cc:348] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.602288 9131 tablet_bootstrap.cc:654] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.602289 9133 raft_consensus.cc:374] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.602236 9130 consensus_queue.cc:260] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:28.602324 9133 raft_consensus.cc:729] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.602442 9129 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:28.602429 9133 consensus_queue.cc:260] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.602511 9129 ts_tablet_manager.cc:1367] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240426 09:49:28.602624 9130 ts_tablet_manager.cc:1398] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:28.602715 9133 ts_tablet_manager.cc:1398] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:28.603197 9129 raft_consensus.cc:348] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.603312 9129 raft_consensus.cc:374] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.603353 9129 raft_consensus.cc:729] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.603464 9129 consensus_queue.cc:260] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.603610 9131 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:28.603672 9131 ts_tablet_manager.cc:1367] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:28.603770 9129 ts_tablet_manager.cc:1398] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:28.604085 9131 raft_consensus.cc:348] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.604187 9131 raft_consensus.cc:374] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.604218 9131 raft_consensus.cc:729] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.604313 9131 consensus_queue.cc:260] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.604544 9131 ts_tablet_manager.cc:1398] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:28.605091 9130 tablet_bootstrap.cc:492] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:28.606132 9130 tablet_bootstrap.cc:654] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:28.608320 9130 tablet_bootstrap.cc:492] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:28.608412 9130 ts_tablet_manager.cc:1367] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:28.608774 9130 raft_consensus.cc:348] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.608862 9130 raft_consensus.cc:374] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:28.608894 9130 raft_consensus.cc:729] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:28.609023 9130 consensus_queue.cc:260] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.609261 9130 ts_tablet_manager.cc:1398] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:28.626657 8892 raft_consensus.cc:482] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:28.626807 8892 raft_consensus.cc:504] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:28.627183 8892 leader_election.cc:288] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:28.627454 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42302cab4524e7a85defe4da40be268" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:28.627480 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42302cab4524e7a85defe4da40be268" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:28.627584 7275 raft_consensus.cc:2454] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:28.627610 7011 raft_consensus.cc:2454] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:28.627882 6815 leader_election.cc:302] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:28.628026 8892 raft_consensus.cc:2781] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:28.628088 8892 raft_consensus.cc:482] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:28.628119 8892 raft_consensus.cc:3037] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.629096 8892 raft_consensus.cc:504] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:28.629464 8892 leader_election.cc:288] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:28.629709 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42302cab4524e7a85defe4da40be268" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:28.629813 7275 raft_consensus.cc:3037] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.630532 7275 raft_consensus.cc:2454] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:28.630735 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42302cab4524e7a85defe4da40be268" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:28.630823 6817 leader_election.cc:302] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:28.630862 7011 raft_consensus.cc:3037] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.630990 8892 raft_consensus.cc:2781] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:28.631098 8892 raft_consensus.cc:686] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:28.631233 8892 consensus_queue.cc:237] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:28.631793 7011 raft_consensus.cc:2454] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:28.633263 6731 catalog_manager.cc:5484] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:28.785049 8766 raft_consensus.cc:482] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:28.785162 8766 raft_consensus.cc:504] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.785584 8766 leader_election.cc:288] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:28.785904 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd377032e7cd4b4998424f8685ad5b85" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:28.785935 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd377032e7cd4b4998424f8685ad5b85" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:28.786048 7275 raft_consensus.cc:2454] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:28.786048 7011 raft_consensus.cc:2454] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:28.786321 7077 leader_election.cc:302] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:28.786476 8766 raft_consensus.cc:2781] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:28.786535 8766 raft_consensus.cc:482] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:28.786583 8766 raft_consensus.cc:3037] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.787413 8766 raft_consensus.cc:504] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.787707 8766 leader_election.cc:288] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:28.787981 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd377032e7cd4b4998424f8685ad5b85" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:28.788012 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd377032e7cd4b4998424f8685ad5b85" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:28.788100 7011 raft_consensus.cc:3037] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.788100 7274 raft_consensus.cc:3037] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.788990 7011 raft_consensus.cc:2454] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:28.788990 7274 raft_consensus.cc:2454] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:28.789286 7077 leader_election.cc:302] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:28.789428 8766 raft_consensus.cc:2781] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:28.789516 8766 raft_consensus.cc:686] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:28.789650 8766 consensus_queue.cc:237] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:28.791651 6731 catalog_manager.cc:5484] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:28.863893 9049 raft_consensus.cc:482] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:28.864035 9049 raft_consensus.cc:504] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.864442 9049 leader_election.cc:288] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:28.864801 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "295e6e95d86c47c88a09e47448c93d2e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:28.864835 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "295e6e95d86c47c88a09e47448c93d2e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:28.864933 6882 raft_consensus.cc:2454] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:28.864951 7143 raft_consensus.cc:2454] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:28.865262 7208 leader_election.cc:302] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:28.865417 9049 raft_consensus.cc:2781] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:28.865475 9049 raft_consensus.cc:482] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:28.865512 9049 raft_consensus.cc:3037] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.867753 9049 raft_consensus.cc:504] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.868098 9049 leader_election.cc:288] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:28.868443 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "295e6e95d86c47c88a09e47448c93d2e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:28.868469 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "295e6e95d86c47c88a09e47448c93d2e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:28.868572 6882 raft_consensus.cc:3037] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.868572 7143 raft_consensus.cc:3037] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:28.869544 7143 raft_consensus.cc:2454] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:28.869544 6882 raft_consensus.cc:2454] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:28.869859 7210 leader_election.cc:302] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:28.870018 9049 raft_consensus.cc:2781] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:28.870112 9049 raft_consensus.cc:686] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:28.870266 9049 consensus_queue.cc:237] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:28.873282 6731 catalog_manager.cc:5484] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
W20240426 09:49:29.007752 9145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:29.008060 9145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:29.011070 9145 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240426 09:49:29.022856 8892 consensus_queue.cc:1035] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:29.027602 9070 consensus_queue.cc:1035] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_19'
I20240426 09:49:29.115082 7011 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:29.115165 7274 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:29.115521 8766 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:29.115661 9043 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:29.121711 7143 raft_consensus.cc:1261] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:29.121824 6882 raft_consensus.cc:1261] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:29.122169 9049 consensus_queue.cc:1035] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:29.122323 8584 consensus_queue.cc:1035] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 122.805 ms
time per row: 0.0614026 ms
I20240426 09:49:29.182746 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:40545
--local_ip_for_outbound_sockets=127.0.99.133
--tserver_master_addrs=127.0.99.190:43685
--webserver_port=40677
--webserver_interface=127.0.99.133
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:49:29.291231 9204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:29.291425 9204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:29.291491 9204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:29.294597 9204 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:29.294665 9204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:29.294744 9204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240426 09:49:29.298166 9204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:40545
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=40677
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.9204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:29.299175 9204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:29.300387 9204 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:29.309180 9212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:29.310009 9209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:29.310879 9210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:29.311688 9204 server_base.cc:1034] running on GCE node
I20240426 09:49:29.312072 9204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:29.312634 9204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:29.313822 9204 hybrid_clock.cc:648] HybridClock initialized: now 1714124969313798 us; error 36 us; skew 500 ppm
I20240426 09:49:29.315800 9204 webserver.cc:460] Webserver started at http://127.0.99.133:40677/ using document root <none> and password file <none>
I20240426 09:49:29.316385 9204 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:29.316452 9204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:29.320019 9204 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240426 09:49:29.322098 9218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.323192 9204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:29.323298 9204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "a692602e1a2e4794877e6fb92a7b9962"
format_stamp: "Formatted at 2024-04-26 09:49:13 on dist-test-slave-1zqj"
I20240426 09:49:29.323709 9204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:29.356480 9204 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:29.357110 9204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:29.357266 9204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:29.357795 9204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:29.358734 9204 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:29.358821 9204 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.358888 9204 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:29.358920 9204 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.367244 9204 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:40545
I20240426 09:49:29.367281 9331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:40545 every 8 connection(s)
I20240426 09:49:29.368252 9204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240426 09:49:29.368582 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 9204
I20240426 09:49:29.369009 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:37203
--local_ip_for_outbound_sockets=127.0.99.134
--tserver_master_addrs=127.0.99.190:43685
--webserver_port=37725
--webserver_interface=127.0.99.134
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:49:29.379062 9332 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:29.379307 9332 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:29.379832 9332 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:29.381019 6740 ts_manager.cc:194] Re-registered known tserver with Master: a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:29.381862 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:59227
W20240426 09:49:29.482182 9334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:29.482466 9334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:29.482555 9334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:29.485565 9334 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:29.485653 9334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:29.485750 9334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240426 09:49:29.489208 9334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:37203
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=37725
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.9334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:29.490027 9334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:29.491149 9334 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:29.499998 9340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:29.500772 9343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:29.501653 9334 server_base.cc:1034] running on GCE node
W20240426 09:49:29.502415 9341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:29.502853 9334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:29.503436 9334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:29.504608 9334 hybrid_clock.cc:648] HybridClock initialized: now 1714124969504584 us; error 37 us; skew 500 ppm
I20240426 09:49:29.511059 9334 webserver.cc:460] Webserver started at http://127.0.99.134:37725/ using document root <none> and password file <none>
I20240426 09:49:29.511653 9334 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:29.511730 9334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:29.515691 9334 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20240426 09:49:29.517607 9349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.518612 9334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:29.518711 9334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "3b41d87e7bfb419f8aa7500100129b6c"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:29.519163 9334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:29.536880 9334 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:29.537513 9334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:29.537678 9334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:29.538200 9334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:29.539170 9334 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:29.539227 9334 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.539278 9334 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:29.539306 9334 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.548278 9334 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:37203
I20240426 09:49:29.548291 9462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:37203 every 8 connection(s)
I20240426 09:49:29.549321 9334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240426 09:49:29.555938 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 9334
I20240426 09:49:29.556401 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:36979
--local_ip_for_outbound_sockets=127.0.99.135
--tserver_master_addrs=127.0.99.190:43685
--webserver_port=40803
--webserver_interface=127.0.99.135
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:49:29.557868 9463 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:29.558113 9463 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:29.558640 9463 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:29.559588 6740 ts_manager.cc:194] Re-registered known tserver with Master: 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:29.560154 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:56059
W20240426 09:49:29.668740 9466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:29.668936 9466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:29.668988 9466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:29.672108 9466 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:29.672173 9466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:29.672272 9466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240426 09:49:29.676023 9466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:36979
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=40803
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.9466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:29.677026 9466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:29.678254 9466 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:29.688046 9473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:29.688798 9472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:29.689636 9475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:29.690511 9466 server_base.cc:1034] running on GCE node
I20240426 09:49:29.691000 9466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:29.691622 9466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:29.693207 9466 hybrid_clock.cc:648] HybridClock initialized: now 1714124969693188 us; error 26 us; skew 500 ppm
I20240426 09:49:29.695618 9466 webserver.cc:460] Webserver started at http://127.0.99.135:40803/ using document root <none> and password file <none>
I20240426 09:49:29.696202 9466 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:29.696305 9466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:29.703620 9466 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.002s
I20240426 09:49:29.705417 9481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.706526 9466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:29.706626 9466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:29.707140 9466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:29.719398 9466 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:29.719954 9466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:29.720145 9466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:29.720695 9466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:29.721611 9466 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:29.721665 9466 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.721727 9466 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:29.721752 9466 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.731307 9466 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:36979
I20240426 09:49:29.731336 9594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:36979 every 8 connection(s)
I20240426 09:49:29.732309 9466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240426 09:49:29.740723 9595 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:29.740937 9595 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:29.741374 9595 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:29.742331 6740 ts_manager.cc:194] Re-registered known tserver with Master: b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:29.742461 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 9466
I20240426 09:49:29.742895 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:45059
--local_ip_for_outbound_sockets=127.0.99.136
--tserver_master_addrs=127.0.99.190:43685
--webserver_port=34323
--webserver_interface=127.0.99.136
--builtin_ntp_servers=127.0.99.148:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:49:29.743053 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:52957
W20240426 09:49:29.859913 9598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:49:29.860196 9598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:49:29.860266 9598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:49:29.863544 9598 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:49:29.863623 9598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:49:29.863716 9598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240426 09:49:29.867269 9598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:45059
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=34323
--tserver_master_addrs=127.0.99.190:43685
--never_fsync=true
--heap_profile_path=/tmp/kudu.9598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:49:29.868176 9598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:49:29.869359 9598 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:49:29.878401 9604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:29.879297 9607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:49:29.880116 9605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:49:29.880931 9598 server_base.cc:1034] running on GCE node
I20240426 09:49:29.881309 9598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:49:29.881819 9598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:49:29.882959 9598 hybrid_clock.cc:648] HybridClock initialized: now 1714124969882944 us; error 31 us; skew 500 ppm
I20240426 09:49:29.884661 9598 webserver.cc:460] Webserver started at http://127.0.99.136:34323/ using document root <none> and password file <none>
I20240426 09:49:29.885174 9598 fs_manager.cc:359] Metadata directory not provided
I20240426 09:49:29.885246 9598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:49:29.888228 9598 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240426 09:49:29.889909 9613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.890946 9598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:29.891047 9598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "558c4ecd0ece460d9591aa7a3c799d19"
format_stamp: "Formatted at 2024-04-26 09:49:14 on dist-test-slave-1zqj"
I20240426 09:49:29.891458 9598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:49:29.916396 9598 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:49:29.917026 9598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:49:29.917182 9598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:49:29.917719 9598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:49:29.918588 9598 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:49:29.918637 9598 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.918697 9598 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:49:29.918725 9598 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:49:29.927907 9598 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:45059
I20240426 09:49:29.929101 9598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240426 09:49:29.929445 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 9598
I20240426 09:49:29.929762 9726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:45059 every 8 connection(s)
I20240426 09:49:29.933197 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:29.944615 6993 tablet_service.cc:1467] Processing CreateTablet for tablet dfc6079f3ac949d5b673857a4b858a0f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=eb55ca15c4544622a1394b51ccef26c5]), partition=
I20240426 09:49:29.944888 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfc6079f3ac949d5b673857a4b858a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:29.945652 7255 tablet_service.cc:1467] Processing CreateTablet for tablet dfc6079f3ac949d5b673857a4b858a0f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=eb55ca15c4544622a1394b51ccef26c5]), partition=
I20240426 09:49:29.945883 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfc6079f3ac949d5b673857a4b858a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:29.948364 9727 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43685
I20240426 09:49:29.948606 9727 heartbeater.cc:464] Registering TS with master...
I20240426 09:49:29.948807 9742 tablet_bootstrap.cc:492] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:29.949124 9727 heartbeater.cc:511] Master 127.0.99.190:43685 requested a full tablet report, sending...
I20240426 09:49:29.949833 9742 tablet_bootstrap.cc:654] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:29.950001 6740 ts_manager.cc:194] Re-registered known tserver with Master: 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:29.950699 6740 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:47227
I20240426 09:49:29.951997 9742 tablet_bootstrap.cc:492] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:29.952080 9742 ts_tablet_manager.cc:1367] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:29.952433 9742 raft_consensus.cc:348] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:29.952536 9742 raft_consensus.cc:374] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:29.952564 9742 raft_consensus.cc:729] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:29.952672 9742 consensus_queue.cc:260] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:29.952929 9742 ts_tablet_manager.cc:1398] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:29.956024 9741 tablet_bootstrap.cc:492] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:29.956924 9741 tablet_bootstrap.cc:654] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:29.961375 9266 tablet_service.cc:1467] Processing CreateTablet for tablet dfc6079f3ac949d5b673857a4b858a0f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=eb55ca15c4544622a1394b51ccef26c5]), partition=
I20240426 09:49:29.962105 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfc6079f3ac949d5b673857a4b858a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:29.963763 9741 tablet_bootstrap.cc:492] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:29.963873 9741 ts_tablet_manager.cc:1367] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:49:29.964296 9741 raft_consensus.cc:348] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:29.964479 9741 raft_consensus.cc:374] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:29.964551 9741 raft_consensus.cc:729] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:29.964731 9741 consensus_queue.cc:260] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:29.965019 9741 ts_tablet_manager.cc:1398] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:29.972062 9745 tablet_bootstrap.cc:492] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:29.974040 9745 tablet_bootstrap.cc:654] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:29.974923 9745 log.cc:824] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:29.979600 9745 tablet_bootstrap.cc:492] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:29.979780 9745 ts_tablet_manager.cc:1367] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20240426 09:49:29.982213 9745 raft_consensus.cc:348] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:29.982434 9745 raft_consensus.cc:374] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:29.982501 9745 raft_consensus.cc:729] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:29.982964 9745 consensus_queue.cc:260] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:29.983659 9332 heartbeater.cc:503] Master 127.0.99.190:43685 was elected leader, sending a full tablet report...
I20240426 09:49:29.983783 9745 ts_tablet_manager.cc:1398] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:49:30.037285 8584 raft_consensus.cc:482] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:30.037405 8584 raft_consensus.cc:504] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:30.037858 8584 leader_election.cc:288] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:30.038297 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfc6079f3ac949d5b673857a4b858a0f" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:30.038446 7011 raft_consensus.cc:2454] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:30.038822 7208 leader_election.cc:302] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:30.039017 8584 raft_consensus.cc:2781] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:30.039075 8584 raft_consensus.cc:482] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:30.039129 8584 raft_consensus.cc:3037] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.040223 8584 raft_consensus.cc:504] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:30.040596 8584 leader_election.cc:288] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:30.041024 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfc6079f3ac949d5b673857a4b858a0f" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:30.041146 7011 raft_consensus.cc:3037] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.042142 7011 raft_consensus.cc:2454] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:30.042508 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfc6079f3ac949d5b673857a4b858a0f" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:30.042553 7208 leader_election.cc:302] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:30.042819 9286 raft_consensus.cc:2454] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:30.043146 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfc6079f3ac949d5b673857a4b858a0f" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:30.043249 9285 raft_consensus.cc:3037] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.043406 8584 raft_consensus.cc:2781] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:30.043516 8584 raft_consensus.cc:686] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:30.043664 8584 consensus_queue.cc:237] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:30.044124 9285 raft_consensus.cc:2454] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:30.046355 6739 catalog_manager.cc:5484] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
W20240426 09:49:30.119462 9333 tablet.cc:2369] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:30.155717 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:30.163934 9285 raft_consensus.cc:1261] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:30.164372 7011 raft_consensus.cc:1261] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:30.164687 8584 consensus_queue.cc:1035] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:30.168473 8584 consensus_queue.cc:1035] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:30.172135 6731 catalog_manager.cc:5576] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135) does not have the latest schema for tablet dfc6079f3ac949d5b673857a4b858a0f (table rebalancer_dynamic_table [id=eb55ca15c4544622a1394b51ccef26c5]). Expected version 1 got 0
I20240426 09:49:30.175653 6731 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet dfc6079f3ac949d5b673857a4b858a0f (table rebalancer_dynamic_table [id=eb55ca15c4544622a1394b51ccef26c5]). Expected version 1 got 0
I20240426 09:49:30.177773 9814 tablet.cc:1722] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:30.178532 9814 tablet.cc:1620] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:30.179610 9811 mvcc.cc:205] Tried to move back new op lower bound from 7021055877780848640 to 7021055877302173696. Current Snapshot: MvccSnapshot[applied={T|T < 7021055877780848640}]
I20240426 09:49:30.181164 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=eb55ca15c4544622a1394b51ccef26c5] alter complete (version 1)
I20240426 09:49:30.187218 9813 tablet.cc:1722] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:30.187968 9813 tablet.cc:1620] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:30.192039 6740 catalog_manager.cc:5576] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135) does not have the latest schema for tablet dfc6079f3ac949d5b673857a4b858a0f (table rebalancer_dynamic_table [id=eb55ca15c4544622a1394b51ccef26c5]). Expected version 1 got 0
I20240426 09:49:30.192271 9815 tablet.cc:1722] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:30.196849 9815 tablet.cc:1620] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:30.230304 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:30.237870 9266 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.237974 9529 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.238696 9661 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.241423 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:30.254010 6860 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:30.262375 7255 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:30.275812 9397 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.282516 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:30.287858 9813 tablet.cc:1722] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:30.288211 9813 tablet.cc:1620] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:30.288416 9814 tablet.cc:1722] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:30.288506 9815 tablet.cc:1722] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:30.288681 9814 tablet.cc:1620] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:30.288812 9815 tablet.cc:1620] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:30.294201 6740 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=eb55ca15c4544622a1394b51ccef26c5] alter complete (version 2)
I20240426 09:49:30.396674 6731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:30.402750 9813 tablet.cc:1722] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:30.405504 9815 tablet.cc:1722] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:30.407455 9814 tablet.cc:1722] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:30.410022 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=eb55ca15c4544622a1394b51ccef26c5] alter complete (version 3)
I20240426 09:49:30.429553 6860 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:30.430558 9397 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.431406 9529 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.432989 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:30.433049 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:30.433362 9266 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.434085 6732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240426 09:49:30.434250 6732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240426 09:49:30.435539 6731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:30.444329 6732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 16fb5fe3061842e199528f94fce0f378
I20240426 09:49:30.444700 6732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2
I20240426 09:49:30.444869 6732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 43d2cce5794647129a4533a2fb9da89f
I20240426 09:49:30.445458 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 16fb5fe3061842e199528f94fce0f378 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:30.445523 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:30.445984 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 43d2cce5794647129a4533a2fb9da89f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:30.445982 6860 tablet_service.cc:1514] Processing DeleteTablet for tablet 16fb5fe3061842e199528f94fce0f378 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:30.446002 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 43d2cce5794647129a4533a2fb9da89f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:30.445982 6861 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:30.446532 9893 tablet_replica.cc:330] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:30.446851 9893 raft_consensus.cc:2229] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.447057 9893 raft_consensus.cc:2258] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.448043 9893 ts_tablet_manager.cc:1875] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240426 09:49:30.449169 6815 consensus_peers.cc:485] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:30.451243 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:30.451629 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 16fb5fe3061842e199528f94fce0f378 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:30.452452 9893 ts_tablet_manager.cc:1888] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.452603 9893 log.cc:1196] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/43d2cce5794647129a4533a2fb9da89f
I20240426 09:49:30.453023 9893 ts_tablet_manager.cc:1909] T 43d2cce5794647129a4533a2fb9da89f P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:30.454053 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 43d2cce5794647129a4533a2fb9da89f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:30.454406 9892 tablet_replica.cc:330] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:30.454564 9893 tablet_replica.cc:330] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:30.454742 9893 raft_consensus.cc:2229] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:30.454753 9892 raft_consensus.cc:2229] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.454934 9892 raft_consensus.cc:2258] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.455076 9893 raft_consensus.cc:2258] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.455745 9893 ts_tablet_manager.cc:1875] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.455955 9892 ts_tablet_manager.cc:1875] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.458072 9893 ts_tablet_manager.cc:1888] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.458174 9893 log.cc:1196] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/2d764ebe4e3d4b958d77bcbbc9f8f8d2
I20240426 09:49:30.458263 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 43d2cce5794647129a4533a2fb9da89f (table rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]) successfully deleted
I20240426 09:49:30.458542 9893 ts_tablet_manager.cc:1909] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:30.458716 9892 ts_tablet_manager.cc:1888] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.458890 9892 log.cc:1196] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/16fb5fe3061842e199528f94fce0f378
I20240426 09:49:30.459583 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 1f1b9c46feae4f659dc2eeb8619c5b4e (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:30.459650 9892 ts_tablet_manager.cc:1909] T 16fb5fe3061842e199528f94fce0f378 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:30.459867 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f1b9c46feae4f659dc2eeb8619c5b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.460202 9266 tablet_service.cc:1467] Processing CreateTablet for tablet af4d2322360b43fdbd04aa430f41494e (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:30.460544 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4d2322360b43fdbd04aa430f41494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.460619 9892 tablet_replica.cc:330] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:30.460740 9892 raft_consensus.cc:2229] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.460856 9892 raft_consensus.cc:2258] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.461333 9892 ts_tablet_manager.cc:1875] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.461890 9741 tablet_bootstrap.cc:492] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:30.462703 9741 tablet_bootstrap.cc:654] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.462716 9745 tablet_bootstrap.cc:492] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:30.463485 6861 tablet_service.cc:1467] Processing CreateTablet for tablet 1f1b9c46feae4f659dc2eeb8619c5b4e (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:30.463671 9745 tablet_bootstrap.cc:654] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.463788 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f1b9c46feae4f659dc2eeb8619c5b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.464434 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2 (table rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]) successfully deleted
I20240426 09:49:30.464581 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 16fb5fe3061842e199528f94fce0f378 (table rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]) successfully deleted
I20240426 09:49:30.464715 9745 tablet_bootstrap.cc:492] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:30.464807 9745 ts_tablet_manager.cc:1367] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:30.465081 9892 ts_tablet_manager.cc:1888] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.465145 9892 log.cc:1196] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/2d764ebe4e3d4b958d77bcbbc9f8f8d2
I20240426 09:49:30.465373 9892 ts_tablet_manager.cc:1909] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:30.465312 9745 raft_consensus.cc:348] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.465425 9745 raft_consensus.cc:374] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.465472 9745 raft_consensus.cc:729] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.465615 9745 consensus_queue.cc:260] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.465876 9745 ts_tablet_manager.cc:1398] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:30.466148 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 5e010e7f54b54b04afe4e3b1df0c739f (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:30.466380 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e010e7f54b54b04afe4e3b1df0c739f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.468267 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2 (table rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]) successfully deleted
I20240426 09:49:30.468986 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:30.469684 9902 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:30.471184 9894 tablet_replica.cc:330] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:30.471488 9902 tablet_bootstrap.cc:654] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.471530 9894 raft_consensus.cc:2229] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:30.471920 9894 raft_consensus.cc:2258] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.472997 9894 ts_tablet_manager.cc:1875] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.475106 9894 ts_tablet_manager.cc:1888] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.475224 9894 log.cc:1196] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/16fb5fe3061842e199528f94fce0f378
I20240426 09:49:30.475261 9895 tablet_replica.cc:330] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:30.475545 9894 ts_tablet_manager.cc:1909] T 16fb5fe3061842e199528f94fce0f378 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:30.475643 9895 raft_consensus.cc:2229] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.475876 9895 raft_consensus.cc:2258] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.476699 9895 ts_tablet_manager.cc:1875] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.476738 9894 tablet_replica.cc:330] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:30.476848 9894 raft_consensus.cc:2229] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:30.476851 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 16fb5fe3061842e199528f94fce0f378 (table rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]) successfully deleted
I20240426 09:49:30.477074 9894 raft_consensus.cc:2258] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.477572 9894 ts_tablet_manager.cc:1875] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.479470 9894 ts_tablet_manager.cc:1888] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.479583 9894 log.cc:1196] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/43d2cce5794647129a4533a2fb9da89f
I20240426 09:49:30.479877 9894 ts_tablet_manager.cc:1909] T 43d2cce5794647129a4533a2fb9da89f P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:30.480408 9894 tablet_replica.cc:330] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:30.480577 9894 raft_consensus.cc:2229] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.480731 9894 raft_consensus.cc:2258] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.481024 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 43d2cce5794647129a4533a2fb9da89f (table rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]) successfully deleted
I20240426 09:49:30.481283 9894 ts_tablet_manager.cc:1875] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.482993 9894 ts_tablet_manager.cc:1888] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.483114 9894 log.cc:1196] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/2d764ebe4e3d4b958d77bcbbc9f8f8d2
I20240426 09:49:30.483419 9894 ts_tablet_manager.cc:1909] T 2d764ebe4e3d4b958d77bcbbc9f8f8d2 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:30.486868 9895 ts_tablet_manager.cc:1888] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.486956 9895 log.cc:1196] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/16fb5fe3061842e199528f94fce0f378
I20240426 09:49:30.487291 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 2d764ebe4e3d4b958d77bcbbc9f8f8d2 (table rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]) successfully deleted
I20240426 09:49:30.487568 9895 ts_tablet_manager.cc:1909] T 16fb5fe3061842e199528f94fce0f378 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:30.488832 9895 tablet_replica.cc:330] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:30.488874 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 16fb5fe3061842e199528f94fce0f378 (table rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]) successfully deleted
I20240426 09:49:30.489006 9895 raft_consensus.cc:2229] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.489128 9895 raft_consensus.cc:2258] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.489547 9895 ts_tablet_manager.cc:1875] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.491348 9895 ts_tablet_manager.cc:1888] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.491338 9661 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.491469 9895 log.cc:1196] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/43d2cce5794647129a4533a2fb9da89f
I20240426 09:49:30.491731 9895 ts_tablet_manager.cc:1909] T 43d2cce5794647129a4533a2fb9da89f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:30.492424 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 43d2cce5794647129a4533a2fb9da89f (table rebalance_test_table_0 [id=6074e1b137544c2eb8f839f21bfb79bb]) successfully deleted
I20240426 09:49:30.495466 9901 tablet_bootstrap.cc:492] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:30.497031 9901 tablet_bootstrap.cc:654] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.498270 9741 tablet_bootstrap.cc:492] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:30.498358 9741 ts_tablet_manager.cc:1367] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.000s
I20240426 09:49:30.498735 9741 raft_consensus.cc:348] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:30.498878 9741 raft_consensus.cc:374] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.498905 9741 raft_consensus.cc:729] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.499028 9741 consensus_queue.cc:260] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:30.499284 9741 ts_tablet_manager.cc:1398] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:30.501746 9902 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:30.501849 9902 ts_tablet_manager.cc:1367] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240426 09:49:30.502277 9902 raft_consensus.cc:348] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:30.502382 9902 raft_consensus.cc:374] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.502411 9902 raft_consensus.cc:729] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.502532 9902 consensus_queue.cc:260] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:30.502842 9902 ts_tablet_manager.cc:1398] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:30.506029 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 1f1b9c46feae4f659dc2eeb8619c5b4e (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:30.507998 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 5e010e7f54b54b04afe4e3b1df0c739f (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:30.508569 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f1b9c46feae4f659dc2eeb8619c5b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.510987 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e010e7f54b54b04afe4e3b1df0c739f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.512440 6732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:30.512552 6732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:30.514566 6732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet dfc6079f3ac949d5b673857a4b858a0f
I20240426 09:49:30.515198 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet dfc6079f3ac949d5b673857a4b858a0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:30.515365 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet dfc6079f3ac949d5b673857a4b858a0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:30.516189 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet dfc6079f3ac949d5b673857a4b858a0f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:30.516358 9895 tablet_replica.cc:330] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:30.516499 9895 raft_consensus.cc:2229] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:30.516762 9895 raft_consensus.cc:2258] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.519484 9895 ts_tablet_manager.cc:1875] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.521596 9906 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:30.521628 9895 ts_tablet_manager.cc:1888] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:30.521696 9895 log.cc:1196] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/dfc6079f3ac949d5b673857a4b858a0f
I20240426 09:49:30.522182 9895 ts_tablet_manager.cc:1909] T dfc6079f3ac949d5b673857a4b858a0f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:30.522195 9892 tablet_replica.cc:330] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:30.522356 9892 raft_consensus.cc:2229] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.522492 9892 raft_consensus.cc:2258] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.523181 9892 ts_tablet_manager.cc:1875] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.524143 9906 tablet_bootstrap.cc:654] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.524941 9906 log.cc:824] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:30.525343 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet dfc6079f3ac949d5b673857a4b858a0f (table rebalancer_dynamic_table_new_name [id=eb55ca15c4544622a1394b51ccef26c5]) successfully deleted
I20240426 09:49:30.526715 9906 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:30.526999 9906 ts_tablet_manager.cc:1367] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240426 09:49:30.528429 9661 tablet_service.cc:1467] Processing CreateTablet for tablet af4d2322360b43fdbd04aa430f41494e (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:30.529392 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4d2322360b43fdbd04aa430f41494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.530069 9906 raft_consensus.cc:348] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:30.530280 9906 raft_consensus.cc:374] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.530325 9906 raft_consensus.cc:729] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.530468 9528 tablet_service.cc:1467] Processing CreateTablet for tablet 5e010e7f54b54b04afe4e3b1df0c739f (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:30.531044 9906 consensus_queue.cc:260] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:30.531302 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e010e7f54b54b04afe4e3b1df0c739f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.531894 9906 ts_tablet_manager.cc:1398] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:49:30.532241 9463 heartbeater.cc:503] Master 127.0.99.190:43685 was elected leader, sending a full tablet report...
I20240426 09:49:30.532244 9906 tablet_bootstrap.cc:492] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:30.534489 9529 tablet_service.cc:1467] Processing CreateTablet for tablet af4d2322360b43fdbd04aa430f41494e (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:30.534718 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4d2322360b43fdbd04aa430f41494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.535470 9906 tablet_bootstrap.cc:654] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.536316 9910 tablet_bootstrap.cc:492] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:30.537922 9910 tablet_bootstrap.cc:654] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.538515 9910 log.cc:824] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:30.539398 9906 tablet_bootstrap.cc:492] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:30.539479 9906 ts_tablet_manager.cc:1367] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:30.539906 9906 raft_consensus.cc:348] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:30.539999 9906 raft_consensus.cc:374] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.540030 9906 raft_consensus.cc:729] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.540199 9906 consensus_queue.cc:260] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:30.540572 9906 ts_tablet_manager.cc:1398] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:30.542129 9910 tablet_bootstrap.cc:492] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:30.542372 9910 ts_tablet_manager.cc:1367] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20240426 09:49:30.543802 9892 ts_tablet_manager.cc:1888] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:30.543887 9892 log.cc:1196] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/dfc6079f3ac949d5b673857a4b858a0f
I20240426 09:49:30.544214 9892 ts_tablet_manager.cc:1909] T dfc6079f3ac949d5b673857a4b858a0f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:30.544845 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet dfc6079f3ac949d5b673857a4b858a0f (table rebalancer_dynamic_table_new_name [id=eb55ca15c4544622a1394b51ccef26c5]) successfully deleted
I20240426 09:49:30.545145 9905 tablet_replica.cc:330] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:30.545432 9905 raft_consensus.cc:2229] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.545639 9905 raft_consensus.cc:2258] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.545665 9910 raft_consensus.cc:348] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.545881 9910 raft_consensus.cc:374] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.545938 9910 raft_consensus.cc:729] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.546694 9905 ts_tablet_manager.cc:1875] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.546998 9910 consensus_queue.cc:260] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.548853 9912 tablet_bootstrap.cc:492] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
W20240426 09:49:30.550652 9464 tablet.cc:2369] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:30.551357 9912 tablet_bootstrap.cc:654] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.552095 9912 log.cc:824] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log is configured to *not* fsync() on all Append() calls
I20240426 09:49:30.553076 9901 tablet_bootstrap.cc:492] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:30.553205 9901 ts_tablet_manager.cc:1367] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.000s
I20240426 09:49:30.553304 9905 ts_tablet_manager.cc:1888] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:30.553393 9905 log.cc:1196] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/dfc6079f3ac949d5b673857a4b858a0f
I20240426 09:49:30.553671 9905 ts_tablet_manager.cc:1909] T dfc6079f3ac949d5b673857a4b858a0f P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:30.553615 9901 raft_consensus.cc:348] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:30.553745 9901 raft_consensus.cc:374] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.553777 9901 raft_consensus.cc:729] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.553892 9901 consensus_queue.cc:260] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:30.554167 9901 ts_tablet_manager.cc:1398] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:30.555749 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet dfc6079f3ac949d5b673857a4b858a0f (table rebalancer_dynamic_table_new_name [id=eb55ca15c4544622a1394b51ccef26c5]) successfully deleted
I20240426 09:49:30.557662 9912 tablet_bootstrap.cc:492] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:30.557847 9912 ts_tablet_manager.cc:1367] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20240426 09:49:30.559010 9910 ts_tablet_manager.cc:1398] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20240426 09:49:30.559387 9727 heartbeater.cc:503] Master 127.0.99.190:43685 was elected leader, sending a full tablet report...
I20240426 09:49:30.562237 9912 raft_consensus.cc:348] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.562464 9912 raft_consensus.cc:374] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.562525 9912 raft_consensus.cc:729] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.562976 9912 consensus_queue.cc:260] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.563639 9595 heartbeater.cc:503] Master 127.0.99.190:43685 was elected leader, sending a full tablet report...
I20240426 09:49:30.565016 9912 ts_tablet_manager.cc:1398] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240426 09:49:30.565305 9912 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:30.566273 9912 tablet_bootstrap.cc:654] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.567374 9912 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:30.567457 9912 ts_tablet_manager.cc:1367] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:30.567986 9912 raft_consensus.cc:348] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:30.568095 9912 raft_consensus.cc:374] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.568133 9912 raft_consensus.cc:729] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.568238 9912 consensus_queue.cc:260] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:30.568500 9912 ts_tablet_manager.cc:1398] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:30.616351 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:30.621487 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 477132d66d054d4da3d304658220ca0e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c46aa474341542c7aa8babce8664ec1f]), partition=
I20240426 09:49:30.621701 9528 tablet_service.cc:1467] Processing CreateTablet for tablet 477132d66d054d4da3d304658220ca0e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c46aa474341542c7aa8babce8664ec1f]), partition=
I20240426 09:49:30.621770 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477132d66d054d4da3d304658220ca0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.621927 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477132d66d054d4da3d304658220ca0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.621968 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 477132d66d054d4da3d304658220ca0e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=c46aa474341542c7aa8babce8664ec1f]), partition=
I20240426 09:49:30.622233 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 477132d66d054d4da3d304658220ca0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:30.624470 9912 tablet_bootstrap.cc:492] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:30.624820 9902 tablet_bootstrap.cc:492] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:30.625707 9902 tablet_bootstrap.cc:654] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.625707 9912 tablet_bootstrap.cc:654] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.626353 9745 tablet_bootstrap.cc:492] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:30.627305 9902 tablet_bootstrap.cc:492] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:30.627411 9902 ts_tablet_manager.cc:1367] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:30.627856 9902 raft_consensus.cc:348] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.627965 9902 raft_consensus.cc:374] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.627995 9902 raft_consensus.cc:729] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.628078 9745 tablet_bootstrap.cc:654] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:30.628103 9902 consensus_queue.cc:260] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.628387 9902 ts_tablet_manager.cc:1398] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:30.629591 9912 tablet_bootstrap.cc:492] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:30.629676 9912 ts_tablet_manager.cc:1367] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:30.630172 9912 raft_consensus.cc:348] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.630256 9745 tablet_bootstrap.cc:492] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:30.630270 9912 raft_consensus.cc:374] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.630302 9912 raft_consensus.cc:729] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.630333 9745 ts_tablet_manager.cc:1367] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:49:30.630436 9912 consensus_queue.cc:260] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.630677 9912 ts_tablet_manager.cc:1398] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:30.630759 9745 raft_consensus.cc:348] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.630884 9745 raft_consensus.cc:374] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:30.630923 9745 raft_consensus.cc:729] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:30.631044 9745 consensus_queue.cc:260] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.631314 9745 ts_tablet_manager.cc:1398] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:30.633353 9907 raft_consensus.cc:482] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:30.633445 9907 raft_consensus.cc:504] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:30.633850 9907 leader_election.cc:288] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:30.638628 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e010e7f54b54b04afe4e3b1df0c739f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:30.639008 9417 raft_consensus.cc:2454] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:30.639079 8988 raft_consensus.cc:482] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:30.639161 8988 raft_consensus.cc:504] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:30.639636 8988 leader_election.cc:288] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:30.639915 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f1b9c46feae4f659dc2eeb8619c5b4e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:30.640054 6880 raft_consensus.cc:2454] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:30.641341 6948 leader_election.cc:302] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:30.641381 7077 leader_election.cc:302] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:30.641566 8988 raft_consensus.cc:2781] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:30.641616 8988 raft_consensus.cc:482] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:30.641655 8988 raft_consensus.cc:3037] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.642196 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e010e7f54b54b04afe4e3b1df0c739f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:30.642484 9549 raft_consensus.cc:2454] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:30.642738 9907 raft_consensus.cc:2781] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:30.642822 9907 raft_consensus.cc:482] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:30.642876 9907 raft_consensus.cc:3037] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.642846 8988 raft_consensus.cc:504] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:30.643204 8988 leader_election.cc:288] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:30.643640 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f1b9c46feae4f659dc2eeb8619c5b4e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:30.643754 6880 raft_consensus.cc:3037] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.643853 9907 raft_consensus.cc:504] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:30.644176 9907 leader_election.cc:288] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:30.644696 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e010e7f54b54b04afe4e3b1df0c739f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:30.644742 6880 raft_consensus.cc:2454] T 1f1b9c46feae4f659dc2eeb8619c5b4e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:30.644870 9417 raft_consensus.cc:3037] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.645674 6948 leader_election.cc:302] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:30.645946 9417 raft_consensus.cc:2454] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:30.645992 9467 raft_consensus.cc:2781] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:30.646059 9467 raft_consensus.cc:686] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:30.646200 9467 consensus_queue.cc:237] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:30.647409 7077 leader_election.cc:302] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:30.647560 8766 raft_consensus.cc:2781] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:30.647639 8766 raft_consensus.cc:686] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:30.647747 8766 consensus_queue.cc:237] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:30.648746 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e010e7f54b54b04afe4e3b1df0c739f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:30.648877 9549 raft_consensus.cc:3037] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.650030 9549 raft_consensus.cc:2454] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:30.651239 6734 catalog_manager.cc:5484] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:30.651291 6733 catalog_manager.cc:5484] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:30.652380 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f1b9c46feae4f659dc2eeb8619c5b4e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:30.652520 9416 raft_consensus.cc:3037] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.653463 9416 raft_consensus.cc:2454] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:30.653999 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f1b9c46feae4f659dc2eeb8619c5b4e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:30.654132 9417 raft_consensus.cc:2362] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1. Re-sending same reply.
I20240426 09:49:30.662418 9907 raft_consensus.cc:482] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:30.662541 9907 raft_consensus.cc:504] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.663059 9907 leader_election.cc:288] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:30.663836 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477132d66d054d4da3d304658220ca0e" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:30.663976 9549 raft_consensus.cc:2454] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:30.664269 7077 leader_election.cc:302] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:30.664417 9907 raft_consensus.cc:2781] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:30.664462 9907 raft_consensus.cc:482] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:30.664510 9907 raft_consensus.cc:3037] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.665285 9907 raft_consensus.cc:504] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.665582 9907 leader_election.cc:288] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:30.665894 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477132d66d054d4da3d304658220ca0e" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:30.666007 9549 raft_consensus.cc:3037] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.667506 9549 raft_consensus.cc:2454] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:30.667567 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477132d66d054d4da3d304658220ca0e" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:30.667701 9286 raft_consensus.cc:2454] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:30.667896 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "477132d66d054d4da3d304658220ca0e" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:30.667981 9285 raft_consensus.cc:3037] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.668713 7077 leader_election.cc:302] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:30.668910 9285 raft_consensus.cc:2454] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:30.669272 9907 raft_consensus.cc:2781] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:30.669353 9907 raft_consensus.cc:686] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:30.669473 9907 consensus_queue.cc:237] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.671730 6735 catalog_manager.cc:5484] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
W20240426 09:49:30.682265 9728 tablet.cc:2369] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240426 09:49:30.733759 9596 tablet.cc:2369] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:30.776074 6731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:30.780351 9285 raft_consensus.cc:1261] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:30.780704 9549 raft_consensus.cc:1261] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:30.780896 9907 consensus_queue.cc:1035] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:30.781304 9907 consensus_queue.cc:1035] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:30.784121 9814 tablet.cc:1722] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:30.784471 9814 tablet.cc:1620] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:30.784417 9973 tablet.cc:1722] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:30.785172 9973 tablet.cc:1620] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:30.785709 6731 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet 477132d66d054d4da3d304658220ca0e (table rebalancer_dynamic_table [id=c46aa474341542c7aa8babce8664ec1f]). Expected version 1 got 0
I20240426 09:49:30.786952 6732 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet 477132d66d054d4da3d304658220ca0e (table rebalancer_dynamic_table [id=c46aa474341542c7aa8babce8664ec1f]). Expected version 1 got 0
I20240426 09:49:30.790294 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=c46aa474341542c7aa8babce8664ec1f] alter complete (version 1)
I20240426 09:49:30.791639 9975 tablet.cc:1722] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:30.791857 6731 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet 477132d66d054d4da3d304658220ca0e (table rebalancer_dynamic_table [id=c46aa474341542c7aa8babce8664ec1f]). Expected version 1 got 0
I20240426 09:49:30.792366 9975 tablet.cc:1620] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:30.856716 9917 raft_consensus.cc:482] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:30.856863 9917 raft_consensus.cc:504] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.857853 9917 leader_election.cc:288] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:30.861747 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4d2322360b43fdbd04aa430f41494e" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:30.861923 9285 raft_consensus.cc:2454] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:30.861907 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4d2322360b43fdbd04aa430f41494e" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:30.862233 9669 raft_consensus.cc:2454] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:30.862671 9483 leader_election.cc:302] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:30.862948 9917 raft_consensus.cc:2781] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:30.863005 9917 raft_consensus.cc:482] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:30.863034 9917 raft_consensus.cc:3037] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.863832 9917 raft_consensus.cc:504] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.864192 9917 leader_election.cc:288] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:30.864526 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4d2322360b43fdbd04aa430f41494e" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:30.864570 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4d2322360b43fdbd04aa430f41494e" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:30.864645 9669 raft_consensus.cc:3037] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.864678 9285 raft_consensus.cc:3037] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:30.865481 9285 raft_consensus.cc:2454] T af4d2322360b43fdbd04aa430f41494e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:30.865726 9483 leader_election.cc:302] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:30.865887 9917 raft_consensus.cc:2781] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:30.865981 9669 raft_consensus.cc:2454] T af4d2322360b43fdbd04aa430f41494e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:30.866142 9917 raft_consensus.cc:686] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:30.866426 9917 consensus_queue.cc:237] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:30.868355 6731 catalog_manager.cc:5484] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:30.889070 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:30.889133 6991 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:30.892060 6731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:30.894688 7254 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:30.895234 6861 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:30.901145 9397 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.902513 9266 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.906971 9661 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:30.907840 9973 tablet.cc:1722] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:30.908157 9814 tablet.cc:1722] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:30.908210 9973 tablet.cc:1620] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:30.908453 9814 tablet.cc:1620] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:30.912159 9975 tablet.cc:1722] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:30.912436 9975 tablet.cc:1620] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:30.919838 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=c46aa474341542c7aa8babce8664ec1f] alter complete (version 2)
I20240426 09:49:30.934515 9528 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:30.964570 6731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240426 09:49:30.964718 6731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240426 09:49:30.967147 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 565d9e485cf144de8c0018f64aacb8b4
I20240426 09:49:30.967451 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 72e2daf3e0d84a12bad5ff76fdd01c88
I20240426 09:49:30.967685 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 8786902a28ee47a1995f77e58e18aa5d
I20240426 09:49:30.967741 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 565d9e485cf144de8c0018f64aacb8b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:30.967758 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 565d9e485cf144de8c0018f64aacb8b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:30.967965 9892 tablet_replica.cc:330] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:30.968111 9892 raft_consensus.cc:2229] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:30.968325 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 72e2daf3e0d84a12bad5ff76fdd01c88 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:30.968413 9892 raft_consensus.cc:2258] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.968444 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 8786902a28ee47a1995f77e58e18aa5d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:30.968953 9892 ts_tablet_manager.cc:1875] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.968950 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 8786902a28ee47a1995f77e58e18aa5d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:30.970968 9895 tablet_replica.cc:330] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:30.971112 9895 raft_consensus.cc:2229] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.971163 9892 ts_tablet_manager.cc:1888] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.971215 9895 raft_consensus.cc:2258] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.971228 9892 log.cc:1196] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/565d9e485cf144de8c0018f64aacb8b4
I20240426 09:49:30.971494 9892 ts_tablet_manager.cc:1909] T 565d9e485cf144de8c0018f64aacb8b4 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:30.971693 9895 ts_tablet_manager.cc:1875] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.972587 6861 tablet_service.cc:1514] Processing DeleteTablet for tablet 565d9e485cf144de8c0018f64aacb8b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:30.972759 6860 tablet_service.cc:1514] Processing DeleteTablet for tablet 8786902a28ee47a1995f77e58e18aa5d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:30.972911 9894 tablet_replica.cc:330] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:30.972893 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 72e2daf3e0d84a12bad5ff76fdd01c88 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:30.973073 9894 raft_consensus.cc:2229] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.973237 9894 raft_consensus.cc:2258] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.973726 9894 ts_tablet_manager.cc:1875] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.973768 9895 ts_tablet_manager.cc:1888] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.973824 9895 log.cc:1196] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/565d9e485cf144de8c0018f64aacb8b4
I20240426 09:49:30.973974 9892 tablet_replica.cc:330] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:30.974089 9892 raft_consensus.cc:2229] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:30.974102 9895 ts_tablet_manager.cc:1909] T 565d9e485cf144de8c0018f64aacb8b4 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:30.974231 9892 raft_consensus.cc:2258] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.974690 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 72e2daf3e0d84a12bad5ff76fdd01c88 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:30 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:30.974805 9892 ts_tablet_manager.cc:1875] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.974936 9895 tablet_replica.cc:330] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:30.975051 9895 raft_consensus.cc:2229] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.975175 9895 raft_consensus.cc:2258] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.975615 9895 ts_tablet_manager.cc:1875] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.975858 9894 ts_tablet_manager.cc:1888] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.975924 9894 log.cc:1196] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/565d9e485cf144de8c0018f64aacb8b4
I20240426 09:49:30.976336 9894 ts_tablet_manager.cc:1909] T 565d9e485cf144de8c0018f64aacb8b4 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:30.976640 9892 ts_tablet_manager.cc:1888] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.976697 9892 log.cc:1196] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/8786902a28ee47a1995f77e58e18aa5d
I20240426 09:49:30.977059 9892 ts_tablet_manager.cc:1909] T 8786902a28ee47a1995f77e58e18aa5d P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:30.977761 9895 ts_tablet_manager.cc:1888] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.977826 9895 log.cc:1196] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/8786902a28ee47a1995f77e58e18aa5d
I20240426 09:49:30.978062 9895 ts_tablet_manager.cc:1909] T 8786902a28ee47a1995f77e58e18aa5d P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:30.978683 9895 tablet_replica.cc:330] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:30.978811 9895 raft_consensus.cc:2229] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:30.979004 9895 raft_consensus.cc:2258] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.979498 9895 ts_tablet_manager.cc:1875] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.979676 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 565d9e485cf144de8c0018f64aacb8b4 (table rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]) successfully deleted
I20240426 09:49:30.979873 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 8786902a28ee47a1995f77e58e18aa5d (table rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]) successfully deleted
I20240426 09:49:30.980280 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 565d9e485cf144de8c0018f64aacb8b4 (table rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]) successfully deleted
I20240426 09:49:30.980432 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 8786902a28ee47a1995f77e58e18aa5d (table rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]) successfully deleted
I20240426 09:49:30.981313 9895 ts_tablet_manager.cc:1888] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.981384 9895 log.cc:1196] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/72e2daf3e0d84a12bad5ff76fdd01c88
I20240426 09:49:30.981526 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 565d9e485cf144de8c0018f64aacb8b4 (table rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]) successfully deleted
I20240426 09:49:30.981642 9895 ts_tablet_manager.cc:1909] T 72e2daf3e0d84a12bad5ff76fdd01c88 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:30.982298 9991 tablet_replica.cc:330] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:30.982456 9991 raft_consensus.cc:2229] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.982601 9991 raft_consensus.cc:2258] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.982630 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 72e2daf3e0d84a12bad5ff76fdd01c88 (table rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]) successfully deleted
I20240426 09:49:30.982795 9894 tablet_replica.cc:330] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:30.982946 9894 raft_consensus.cc:2229] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.983062 9894 raft_consensus.cc:2258] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.983145 9991 ts_tablet_manager.cc:1875] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.983521 9894 ts_tablet_manager.cc:1875] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.985291 9991 ts_tablet_manager.cc:1888] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.985371 9991 log.cc:1196] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/72e2daf3e0d84a12bad5ff76fdd01c88
I20240426 09:49:30.985553 9894 ts_tablet_manager.cc:1888] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.985628 9894 log.cc:1196] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/8786902a28ee47a1995f77e58e18aa5d
I20240426 09:49:30.985687 9991 ts_tablet_manager.cc:1909] T 72e2daf3e0d84a12bad5ff76fdd01c88 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:30.985884 9894 ts_tablet_manager.cc:1909] T 8786902a28ee47a1995f77e58e18aa5d P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:30.986402 9894 tablet_replica.cc:330] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:30.986572 9894 raft_consensus.cc:2229] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:30.986716 9894 raft_consensus.cc:2258] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:30.986795 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 72e2daf3e0d84a12bad5ff76fdd01c88 (table rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]) successfully deleted
I20240426 09:49:30.987097 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 8786902a28ee47a1995f77e58e18aa5d (table rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]) successfully deleted
I20240426 09:49:30.987241 9894 ts_tablet_manager.cc:1875] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:30.988857 9894 ts_tablet_manager.cc:1888] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:30.988917 9894 log.cc:1196] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/72e2daf3e0d84a12bad5ff76fdd01c88
I20240426 09:49:30.989120 9894 ts_tablet_manager.cc:1909] T 72e2daf3e0d84a12bad5ff76fdd01c88 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:30.989635 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 72e2daf3e0d84a12bad5ff76fdd01c88 (table rebalance_test_table_2 [id=6e2cb8e5637948ed913944ab6e1ed0df]) successfully deleted
I20240426 09:49:31.016012 7274 consensus_queue.cc:237] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:31.017505 6880 raft_consensus.cc:1261] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.017930 7142 raft_consensus.cc:1261] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.017901 9049 consensus_queue.cc:1035] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.018342 10008 consensus_queue.cc:1035] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.020614 10008 raft_consensus.cc:2932] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.021534 6731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
W20240426 09:49:31.022576 7208 consensus_peers.cc:485] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f3b2b3e1d02b49269d19d284cab418da. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.023200 6880 raft_consensus.cc:2932] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.024713 7142 raft_consensus.cc:2932] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.026522 8988 consensus_queue.cc:1035] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.028366 6735 catalog_manager.cc:5484] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.030894 9973 tablet.cc:1722] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:31.032050 9814 tablet.cc:1722] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:31.034941 9975 tablet.cc:1722] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:31.036419 6740 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=c46aa474341542c7aa8babce8664ec1f] alter complete (version 3)
I20240426 09:49:31.037062 8988 consensus_queue.cc:1035] T 1f1b9c46feae4f659dc2eeb8619c5b4e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.056747 7013 consensus_queue.cc:237] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:31.058233 6880 raft_consensus.cc:1261] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.058269 7274 raft_consensus.cc:1261] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:31.058619 6946 consensus_peers.cc:485] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4bb42dad9a04eb78e0ee6453c8693b7. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.058646 10033 consensus_queue.cc:1035] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.058842 9467 consensus_queue.cc:1035] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.060834 9467 raft_consensus.cc:2932] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.061261 7274 raft_consensus.cc:2932] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.063553 6880 raft_consensus.cc:2932] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.064190 6732 catalog_manager.cc:5484] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.085830 6880 consensus_queue.cc:237] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.087646 7142 raft_consensus.cc:1261] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.088037 7013 raft_consensus.cc:1261] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.088419 10053 consensus_queue.cc:1035] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.088757 9749 consensus_queue.cc:1035] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.090696 9749 raft_consensus.cc:2932] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.091084 7013 raft_consensus.cc:2932] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
W20240426 09:49:31.091414 6816 consensus_peers.cc:485] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6dc287ff7ea640a38537c74e9ff156f7. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.092687 7142 raft_consensus.cc:2932] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.095130 6732 catalog_manager.cc:5484] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.112323 7274 consensus_queue.cc:237] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:31.113977 6880 raft_consensus.cc:1261] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.114068 7013 raft_consensus.cc:1261] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.114378 10008 consensus_queue.cc:1035] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.114665 8584 consensus_queue.cc:1035] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.116598 10008 raft_consensus.cc:2932] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.117199 7013 raft_consensus.cc:2932] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
W20240426 09:49:31.118746 7208 consensus_peers.cc:485] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce1e72dc91434c4ab63639fcc50da2ec. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.119954 6731 catalog_manager.cc:5484] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.121627 6880 raft_consensus.cc:2932] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.139351 7013 consensus_queue.cc:237] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:31.141011 6880 raft_consensus.cc:1261] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.141105 6740 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:31.141233 6740 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:31.141535 7274 raft_consensus.cc:1261] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.141638 10033 consensus_queue.cc:1035] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.142529 10033 consensus_queue.cc:1035] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.143553 6740 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 477132d66d054d4da3d304658220ca0e
I20240426 09:49:31.144507 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 477132d66d054d4da3d304658220ca0e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:31.144618 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 477132d66d054d4da3d304658220ca0e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:31.144883 9528 tablet_service.cc:1514] Processing DeleteTablet for tablet 477132d66d054d4da3d304658220ca0e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:31.144896 10036 raft_consensus.cc:2932] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.145298 9991 tablet_replica.cc:330] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:31.145311 10095 tablet_replica.cc:330] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:31.145438 9991 raft_consensus.cc:2229] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:31.145345 7274 raft_consensus.cc:2932] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.145578 10095 raft_consensus.cc:2229] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.145627 9991 raft_consensus.cc:2258] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.145949 10095 raft_consensus.cc:2258] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
W20240426 09:49:31.145979 6946 consensus_peers.cc:485] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bcbf45071dd240c2ab6e75a9beac5b31. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.146464 9991 ts_tablet_manager.cc:1875] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.147145 10095 ts_tablet_manager.cc:1875] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.148658 9991 ts_tablet_manager.cc:1888] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:31.148727 9991 log.cc:1196] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/477132d66d054d4da3d304658220ca0e
I20240426 09:49:31.149003 9991 ts_tablet_manager.cc:1909] T 477132d66d054d4da3d304658220ca0e P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:31.149580 10095 ts_tablet_manager.cc:1888] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:31.149657 10095 log.cc:1196] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/477132d66d054d4da3d304658220ca0e
I20240426 09:49:31.150019 10095 ts_tablet_manager.cc:1909] T 477132d66d054d4da3d304658220ca0e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:31.150913 6740 catalog_manager.cc:5484] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.151269 10094 tablet_replica.cc:330] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:31.151386 10094 raft_consensus.cc:2229] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.152259 10094 raft_consensus.cc:2258] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.152268 6880 raft_consensus.cc:2932] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.151021 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 477132d66d054d4da3d304658220ca0e (table rebalancer_dynamic_table_new_name [id=c46aa474341542c7aa8babce8664ec1f]) successfully deleted
I20240426 09:49:31.155154 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 477132d66d054d4da3d304658220ca0e (table rebalancer_dynamic_table_new_name [id=c46aa474341542c7aa8babce8664ec1f]) successfully deleted
I20240426 09:49:31.155838 10094 ts_tablet_manager.cc:1875] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.158385 10094 ts_tablet_manager.cc:1888] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:31.158471 10094 log.cc:1196] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/477132d66d054d4da3d304658220ca0e
I20240426 09:49:31.158856 10094 ts_tablet_manager.cc:1909] T 477132d66d054d4da3d304658220ca0e P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:31.159471 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 477132d66d054d4da3d304658220ca0e (table rebalancer_dynamic_table_new_name [id=c46aa474341542c7aa8babce8664ec1f]) successfully deleted
I20240426 09:49:31.178257 7142 consensus_queue.cc:237] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:31.179692 7274 raft_consensus.cc:1261] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.179694 7013 raft_consensus.cc:1261] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.180073 10112 consensus_queue.cc:1035] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:49:31.180253 7077 consensus_peers.cc:485] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0f67d94463ce46708246808da0b100e9. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.180262 8766 consensus_queue.cc:1035] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.181974 8766 raft_consensus.cc:2932] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.182423 7013 raft_consensus.cc:2932] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.183833 7274 raft_consensus.cc:2932] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.186406 6740 catalog_manager.cc:5484] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.191274 8766 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.195314 8766 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.206611 7274 consensus_queue.cc:237] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.207983 7142 raft_consensus.cc:1261] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:31.208189 7209 consensus_peers.cc:485] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a8912e37d7e941a791dd77560e2b3b6b. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.208220 6880 raft_consensus.cc:1261] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.208398 9049 consensus_queue.cc:1035] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.208586 10008 consensus_queue.cc:1035] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.211261 9049 raft_consensus.cc:2932] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.211686 7142 raft_consensus.cc:2932] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.214411 6735 catalog_manager.cc:5484] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.215662 6880 raft_consensus.cc:2932] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.231478 7142 consensus_queue.cc:237] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:31.232822 7011 raft_consensus.cc:1261] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:31.232916 7077 consensus_peers.cc:485] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aca60cffb8cf41e3b0d9b64436c306d7. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.232919 6880 raft_consensus.cc:1261] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.233283 8766 consensus_queue.cc:1035] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.233479 9907 consensus_queue.cc:1035] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.235337 8766 raft_consensus.cc:2932] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.235690 6880 raft_consensus.cc:2932] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.238358 6731 catalog_manager.cc:5484] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.239552 7011 raft_consensus.cc:2932] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.245568 6735 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:31.250530 9661 tablet_service.cc:1467] Processing CreateTablet for tablet f15176ddc6654756bc8042b3e66cacf6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ff8716e2cc294ac0ac631a2db4575f63]), partition=
I20240426 09:49:31.250600 9528 tablet_service.cc:1467] Processing CreateTablet for tablet f15176ddc6654756bc8042b3e66cacf6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ff8716e2cc294ac0ac631a2db4575f63]), partition=
I20240426 09:49:31.250813 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15176ddc6654756bc8042b3e66cacf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.250912 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15176ddc6654756bc8042b3e66cacf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.252996 10159 tablet_bootstrap.cc:492] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:31.253348 7254 tablet_service.cc:1467] Processing CreateTablet for tablet f15176ddc6654756bc8042b3e66cacf6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ff8716e2cc294ac0ac631a2db4575f63]), partition=
I20240426 09:49:31.253578 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f15176ddc6654756bc8042b3e66cacf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.254015 10159 tablet_bootstrap.cc:654] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.254061 10160 tablet_bootstrap.cc:492] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:31.254978 10160 tablet_bootstrap.cc:654] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.255056 10159 tablet_bootstrap.cc:492] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:31.255134 10159 ts_tablet_manager.cc:1367] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:31.255620 10159 raft_consensus.cc:348] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.255728 10159 raft_consensus.cc:374] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.255757 10159 raft_consensus.cc:729] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.255879 10159 consensus_queue.cc:260] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.255991 10160 tablet_bootstrap.cc:492] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:31.256062 10160 ts_tablet_manager.cc:1367] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:31.256139 10159 ts_tablet_manager.cc:1398] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:31.256626 10160 raft_consensus.cc:348] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.256722 10160 raft_consensus.cc:374] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.256748 10160 raft_consensus.cc:729] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.256858 10160 consensus_queue.cc:260] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.257139 10160 ts_tablet_manager.cc:1398] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:31.257177 10163 tablet_bootstrap.cc:492] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:31.257952 10163 tablet_bootstrap.cc:654] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.258904 10163 tablet_bootstrap.cc:492] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:31.258996 10163 ts_tablet_manager.cc:1367] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:31.259388 10163 raft_consensus.cc:348] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.259488 10163 raft_consensus.cc:374] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.259519 10163 raft_consensus.cc:729] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.259629 10163 consensus_queue.cc:260] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.259896 10163 ts_tablet_manager.cc:1398] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:31.312052 9978 consensus_queue.cc:1035] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.318056 9978 consensus_queue.cc:1035] T af4d2322360b43fdbd04aa430f41494e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.324463 9978 raft_consensus.cc:482] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:31.324579 9978 raft_consensus.cc:504] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.325026 9978 leader_election.cc:288] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:31.325723 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15176ddc6654756bc8042b3e66cacf6" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:31.325870 9669 raft_consensus.cc:2454] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:31.326351 9482 leader_election.cc:302] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:31.327639 9978 raft_consensus.cc:2781] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:31.327697 9978 raft_consensus.cc:482] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:31.327728 9978 raft_consensus.cc:3037] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.329052 9978 raft_consensus.cc:504] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.329355 9978 leader_election.cc:288] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:31.329584 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15176ddc6654756bc8042b3e66cacf6" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:31.329689 9669 raft_consensus.cc:3037] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.330581 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15176ddc6654756bc8042b3e66cacf6" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:31.330632 9669 raft_consensus.cc:2454] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:31.330718 7274 raft_consensus.cc:2454] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:31.330958 9482 leader_election.cc:302] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:31.330952 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f15176ddc6654756bc8042b3e66cacf6" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:31.331048 7275 raft_consensus.cc:3037] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.332008 7275 raft_consensus.cc:2454] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:31.332484 9978 raft_consensus.cc:2781] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:31.332561 9978 raft_consensus.cc:686] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:31.332698 9978 consensus_queue.cc:237] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.335661 6731 catalog_manager.cc:5484] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:31.358247 9266 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:31.358690 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:31.362941 6991 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:31.370805 9397 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:31.374989 6862 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:31.378019 9661 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:31.379058 9528 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:31.383173 7254 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:31.434935 6731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:31.441424 7254 tablet_service.cc:1467] Processing CreateTablet for tablet e856ff2c8ffb4bd4b44c694047c8a8fa (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:31.441767 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e856ff2c8ffb4bd4b44c694047c8a8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.442018 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 722306427b7b46b4b0942193c821d990 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:31.442265 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722306427b7b46b4b0942193c821d990. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.442606 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 722306427b7b46b4b0942193c821d990 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:31.442909 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722306427b7b46b4b0942193c821d990. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.444204 9528 tablet_service.cc:1467] Processing CreateTablet for tablet e856ff2c8ffb4bd4b44c694047c8a8fa (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:31.444422 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e856ff2c8ffb4bd4b44c694047c8a8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.444447 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 852fad44c59f42dd9ce66b84819a9296 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:31.444717 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 852fad44c59f42dd9ce66b84819a9296. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.445163 10159 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:31.446127 10159 tablet_bootstrap.cc:654] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.446381 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 852fad44c59f42dd9ce66b84819a9296 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:31.446586 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 852fad44c59f42dd9ce66b84819a9296. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.447721 9661 tablet_service.cc:1467] Processing CreateTablet for tablet e856ff2c8ffb4bd4b44c694047c8a8fa (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:31.448012 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e856ff2c8ffb4bd4b44c694047c8a8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.448369 10159 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:31.448444 10159 ts_tablet_manager.cc:1367] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:31.448546 10163 tablet_bootstrap.cc:492] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:31.448913 10159 raft_consensus.cc:348] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.449020 10159 raft_consensus.cc:374] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.449059 10159 raft_consensus.cc:729] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.449187 10159 consensus_queue.cc:260] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.449453 10159 ts_tablet_manager.cc:1398] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:31.449518 10163 tablet_bootstrap.cc:654] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.450058 10160 tablet_bootstrap.cc:492] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:31.450315 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 852fad44c59f42dd9ce66b84819a9296 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:31.450554 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 852fad44c59f42dd9ce66b84819a9296. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.451313 10160 tablet_bootstrap.cc:654] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.452642 10163 tablet_bootstrap.cc:492] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:31.452730 10163 ts_tablet_manager.cc:1367] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240426 09:49:31.452734 9659 tablet_service.cc:1467] Processing CreateTablet for tablet 722306427b7b46b4b0942193c821d990 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:31.452955 9659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722306427b7b46b4b0942193c821d990. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.453028 10159 tablet_bootstrap.cc:492] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:31.453145 10163 raft_consensus.cc:348] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.453254 10163 raft_consensus.cc:374] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.453287 10163 raft_consensus.cc:729] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.453404 10163 consensus_queue.cc:260] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.453693 10163 ts_tablet_manager.cc:1398] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:31.453850 10163 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:31.454008 10159 tablet_bootstrap.cc:654] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.454704 10163 tablet_bootstrap.cc:654] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.457357 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:31.457998 10160 tablet_bootstrap.cc:492] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:31.458101 10160 ts_tablet_manager.cc:1367] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:31.458532 10160 raft_consensus.cc:348] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.458647 10160 raft_consensus.cc:374] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.458691 10160 raft_consensus.cc:729] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.458830 10160 consensus_queue.cc:260] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.459120 10160 ts_tablet_manager.cc:1398] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:31.459471 10215 tablet_bootstrap.cc:492] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:31.459689 10160 tablet_bootstrap.cc:492] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:31.459754 10163 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:31.459843 10163 ts_tablet_manager.cc:1367] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:31.460315 10163 raft_consensus.cc:348] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.460433 10163 raft_consensus.cc:374] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.460471 10163 raft_consensus.cc:729] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.460530 10215 tablet_bootstrap.cc:654] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.460592 10163 consensus_queue.cc:260] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.460882 10163 ts_tablet_manager.cc:1398] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:31.461910 10215 tablet_bootstrap.cc:492] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:31.462002 10215 ts_tablet_manager.cc:1367] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:31.462505 10215 raft_consensus.cc:348] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:31.462610 10215 raft_consensus.cc:374] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.462641 10215 raft_consensus.cc:729] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.462733 10215 consensus_queue.cc:260] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:31.463064 10215 ts_tablet_manager.cc:1398] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:31.463196 10160 tablet_bootstrap.cc:654] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.464280 10160 tablet_bootstrap.cc:492] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:31.464365 10160 ts_tablet_manager.cc:1367] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240426 09:49:31.464831 10160 raft_consensus.cc:348] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:31.465632 9669 raft_consensus.cc:1261] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:31.466286 10160 raft_consensus.cc:374] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.466413 10160 raft_consensus.cc:729] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.466431 7275 raft_consensus.cc:1261] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:31.466632 9917 consensus_queue.cc:1035] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.466807 10160 consensus_queue.cc:260] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:31.467229 10160 ts_tablet_manager.cc:1398] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:31.467446 10160 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:31.468046 10159 tablet_bootstrap.cc:492] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:31.468150 10159 ts_tablet_manager.cc:1367] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20240426 09:49:31.468505 10160 tablet_bootstrap.cc:654] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.468700 10159 raft_consensus.cc:348] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.468827 10159 raft_consensus.cc:374] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.468912 10159 raft_consensus.cc:729] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.469064 10159 consensus_queue.cc:260] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.469341 10159 ts_tablet_manager.cc:1398] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:31.469439 9978 consensus_queue.cc:1035] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.469529 10159 tablet_bootstrap.cc:492] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:31.470466 10159 tablet_bootstrap.cc:654] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.470851 9975 tablet.cc:1722] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:31.471138 9975 tablet.cc:1620] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:31.472522 6731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240426 09:49:31.472651 6731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240426 09:49:31.472821 10200 mvcc.cc:205] Tried to move back new op lower bound from 7021055883118223360 to 7021055882580066304. Current Snapshot: MvccSnapshot[applied={T|T < 7021055883118223360}]
I20240426 09:49:31.473659 6732 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet f15176ddc6654756bc8042b3e66cacf6 (table rebalancer_dynamic_table [id=ff8716e2cc294ac0ac631a2db4575f63]). Expected version 1 got 0
I20240426 09:49:31.474068 10160 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:31.474149 10160 ts_tablet_manager.cc:1367] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240426 09:49:31.474639 10160 raft_consensus.cc:348] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.474740 10160 raft_consensus.cc:374] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.474771 10160 raft_consensus.cc:729] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.474915 10160 consensus_queue.cc:260] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.475376 10160 ts_tablet_manager.cc:1398] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:31.477108 6740 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet f15176ddc6654756bc8042b3e66cacf6 (table rebalancer_dynamic_table [id=ff8716e2cc294ac0ac631a2db4575f63]). Expected version 1 got 0
I20240426 09:49:31.477180 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 3f2e6050128b45f3b6866a6770ebafb8
I20240426 09:49:31.477483 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet bcb2d524281d43e4bbb962ed7b71e330
I20240426 09:49:31.477736 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 015b439880d94914b0571789407be4f9
I20240426 09:49:31.477725 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2e6050128b45f3b6866a6770ebafb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:31.477878 6739 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet f15176ddc6654756bc8042b3e66cacf6 (table rebalancer_dynamic_table [id=ff8716e2cc294ac0ac631a2db4575f63]). Expected version 1 got 0
I20240426 09:49:31.477916 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2e6050128b45f3b6866a6770ebafb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:31.478070 9991 tablet_replica.cc:330] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:31.478132 9892 tablet_replica.cc:330] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:31.478260 9991 raft_consensus.cc:2229] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:31.478284 9892 raft_consensus.cc:2229] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.478425 9892 raft_consensus.cc:2258] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.478489 9991 raft_consensus.cc:2258] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.478688 6860 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb2d524281d43e4bbb962ed7b71e330 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:31.478855 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2e6050128b45f3b6866a6770ebafb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:31.478919 9894 tablet_replica.cc:330] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:31.478855 6861 tablet_service.cc:1514] Processing DeleteTablet for tablet 015b439880d94914b0571789407be4f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:31.479025 9892 ts_tablet_manager.cc:1875] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.479069 9894 raft_consensus.cc:2229] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.479135 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 015b439880d94914b0571789407be4f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:31.479223 9894 raft_consensus.cc:2258] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.479269 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb2d524281d43e4bbb962ed7b71e330 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:31.479503 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=ff8716e2cc294ac0ac631a2db4575f63] alter complete (version 1)
I20240426 09:49:31.479722 9894 ts_tablet_manager.cc:1875] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.479722 9991 ts_tablet_manager.cc:1875] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.481377 9892 ts_tablet_manager.cc:1888] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:31.481448 9892 log.cc:1196] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/3f2e6050128b45f3b6866a6770ebafb8
I20240426 09:49:31.481707 9892 ts_tablet_manager.cc:1909] T 3f2e6050128b45f3b6866a6770ebafb8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:31.481791 9894 ts_tablet_manager.cc:1888] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:31.481869 9894 log.cc:1196] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/bcb2d524281d43e4bbb962ed7b71e330
I20240426 09:49:31.481858 9991 ts_tablet_manager.cc:1888] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:31.481920 9991 log.cc:1196] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/3f2e6050128b45f3b6866a6770ebafb8
I20240426 09:49:31.482137 9894 ts_tablet_manager.cc:1909] T bcb2d524281d43e4bbb962ed7b71e330 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:31.482177 9991 ts_tablet_manager.cc:1909] T 3f2e6050128b45f3b6866a6770ebafb8 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:31.482307 9892 tablet_replica.cc:330] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:31.482419 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet bcb2d524281d43e4bbb962ed7b71e330 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:31.482461 9892 raft_consensus.cc:2229] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.482580 9892 raft_consensus.cc:2258] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.482448 10231 tablet.cc:1722] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:31.482908 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 015b439880d94914b0571789407be4f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:31.483327 9894 tablet_replica.cc:330] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:31.483446 9894 raft_consensus.cc:2229] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.483439 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet bcb2d524281d43e4bbb962ed7b71e330 (table rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]) successfully deleted
I20240426 09:49:31.483670 10231 tablet.cc:1620] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:31.483827 9991 tablet_replica.cc:330] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:31.483822 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 3f2e6050128b45f3b6866a6770ebafb8 (table rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]) successfully deleted
I20240426 09:49:31.483961 9991 raft_consensus.cc:2229] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.484076 9991 raft_consensus.cc:2258] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.484753 9991 ts_tablet_manager.cc:1875] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.485654 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 3f2e6050128b45f3b6866a6770ebafb8 (table rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]) successfully deleted
I20240426 09:49:31.485836 9894 raft_consensus.cc:2258] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.486289 6739 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet f15176ddc6654756bc8042b3e66cacf6 (table rebalancer_dynamic_table [id=ff8716e2cc294ac0ac631a2db4575f63]). Expected version 1 got 0
I20240426 09:49:31.486348 9892 ts_tablet_manager.cc:1875] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.486476 9894 ts_tablet_manager.cc:1875] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.487489 9991 ts_tablet_manager.cc:1888] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:31.487622 9991 log.cc:1196] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/015b439880d94914b0571789407be4f9
I20240426 09:49:31.487973 9991 ts_tablet_manager.cc:1909] T 015b439880d94914b0571789407be4f9 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:31.488510 9892 ts_tablet_manager.cc:1888] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:31.488590 9892 log.cc:1196] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/bcb2d524281d43e4bbb962ed7b71e330
I20240426 09:49:31.488879 9892 ts_tablet_manager.cc:1909] T bcb2d524281d43e4bbb962ed7b71e330 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:31.489449 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 015b439880d94914b0571789407be4f9 (table rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]) successfully deleted
I20240426 09:49:31.489724 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet bcb2d524281d43e4bbb962ed7b71e330 (table rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]) successfully deleted
I20240426 09:49:31.490713 9894 ts_tablet_manager.cc:1888] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:31.490806 9894 log.cc:1196] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/3f2e6050128b45f3b6866a6770ebafb8
I20240426 09:49:31.491055 10233 ts_tablet_manager.cc:902] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:31.491080 9894 ts_tablet_manager.cc:1909] T 3f2e6050128b45f3b6866a6770ebafb8 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:31.491256 10238 tablet_replica.cc:330] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:31.491394 10238 raft_consensus.cc:2229] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:31.491580 10238 raft_consensus.cc:2258] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.491649 9894 tablet_replica.cc:330] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:31.491744 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 3f2e6050128b45f3b6866a6770ebafb8 (table rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]) successfully deleted
I20240426 09:49:31.491835 10233 tablet_copy_client.cc:308] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:31.491870 9894 raft_consensus.cc:2229] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.492071 9894 raft_consensus.cc:2258] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.492161 10238 ts_tablet_manager.cc:1875] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.492225 10232 tablet.cc:1722] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:31.492547 10232 tablet.cc:1620] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:31.494179 10238 ts_tablet_manager.cc:1888] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:31.494247 10238 log.cc:1196] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/bcb2d524281d43e4bbb962ed7b71e330
I20240426 09:49:31.494982 9894 ts_tablet_manager.cc:1875] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.495589 10238 ts_tablet_manager.cc:1909] T bcb2d524281d43e4bbb962ed7b71e330 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:31.496932 10238 tablet_replica.cc:330] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:31.497031 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet bcb2d524281d43e4bbb962ed7b71e330 (table rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]) successfully deleted
I20240426 09:49:31.497128 10238 raft_consensus.cc:2229] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:31.497341 10238 raft_consensus.cc:2258] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.497534 9894 ts_tablet_manager.cc:1888] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:31.497653 9894 log.cc:1196] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/015b439880d94914b0571789407be4f9
I20240426 09:49:31.497917 10238 ts_tablet_manager.cc:1875] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.498123 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 6dc287ff7ea640a38537c74e9ff156f7 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:54839)
I20240426 09:49:31.498251 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 6dc287ff7ea640a38537c74e9ff156f7 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:54839: session id = a692602e1a2e4794877e6fb92a7b9962-6dc287ff7ea640a38537c74e9ff156f7
I20240426 09:49:31.498584 9894 ts_tablet_manager.cc:1909] T 015b439880d94914b0571789407be4f9 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:31.499816 6902 tablet_copy_source_session.cc:205] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:31.500013 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 015b439880d94914b0571789407be4f9 (table rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]) successfully deleted
I20240426 09:49:31.500138 8988 raft_consensus.cc:482] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:31.500247 8988 raft_consensus.cc:504] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:31.500412 10238 ts_tablet_manager.cc:1888] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:31.500559 10238 log.cc:1196] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/015b439880d94914b0571789407be4f9
I20240426 09:49:31.500780 8988 leader_election.cc:288] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:31.500907 10238 ts_tablet_manager.cc:1909] T 015b439880d94914b0571789407be4f9 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:31.501561 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "852fad44c59f42dd9ce66b84819a9296" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:31.501716 9669 raft_consensus.cc:2454] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:31.501914 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 015b439880d94914b0571789407be4f9 (table rebalance_test_table_4 [id=ff2d6e2f4d1740eea6037c1d034f680a]) successfully deleted
I20240426 09:49:31.502310 6946 leader_election.cc:302] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:31.502471 8988 raft_consensus.cc:2781] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:31.502549 8988 raft_consensus.cc:482] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:31.502596 8988 raft_consensus.cc:3037] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.503192 10233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc287ff7ea640a38537c74e9ff156f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.504529 8988 raft_consensus.cc:504] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:31.505017 8988 leader_election.cc:288] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:31.505633 10233 tablet_copy_client.cc:783] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:31.505795 10233 tablet_copy_client.cc:647] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:31.506449 10159 tablet_bootstrap.cc:492] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:31.506419 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "852fad44c59f42dd9ce66b84819a9296" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:31.506543 10159 ts_tablet_manager.cc:1367] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.000s
I20240426 09:49:31.506543 9669 raft_consensus.cc:3037] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.507062 10159 raft_consensus.cc:348] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:31.507166 10159 raft_consensus.cc:374] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.507192 10159 raft_consensus.cc:729] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.507298 10159 consensus_queue.cc:260] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:31.507545 10159 ts_tablet_manager.cc:1398] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:31.507570 9669 raft_consensus.cc:2454] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:31.511235 6946 leader_election.cc:302] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:31.511386 8988 raft_consensus.cc:2781] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:31.511463 8988 raft_consensus.cc:686] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:31.511613 8988 consensus_queue.cc:237] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:31.513038 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "852fad44c59f42dd9ce66b84819a9296" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:31.513183 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "852fad44c59f42dd9ce66b84819a9296" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:31.513185 9549 raft_consensus.cc:2454] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:31.515462 10233 tablet_copy_client.cc:523] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:31.516734 6740 catalog_manager.cc:5484] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:31.517658 10233 tablet_bootstrap.cc:492] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:31.524052 7011 consensus_queue.cc:237] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.525736 7275 raft_consensus.cc:1261] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.525977 6882 raft_consensus.cc:1261] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.526384 8988 consensus_queue.cc:1035] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.527050 8988 consensus_queue.cc:1035] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:49:31.530107 6947 consensus_peers.cc:485] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 74c34398941848f5989acb2a1a88a82f. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.535185 8988 raft_consensus.cc:2932] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.538409 7275 raft_consensus.cc:2932] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.538468 6740 catalog_manager.cc:5484] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.539201 6882 raft_consensus.cc:2932] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.547472 8584 raft_consensus.cc:482] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:31.547592 8584 raft_consensus.cc:504] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.548043 8584 leader_election.cc:288] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:31.548418 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722306427b7b46b4b0942193c821d990" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:31.548550 9669 raft_consensus.cc:2454] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:31.548810 7208 leader_election.cc:302] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:31.549038 8584 raft_consensus.cc:2781] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:31.549088 8584 raft_consensus.cc:482] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:31.549126 8584 raft_consensus.cc:3037] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.549844 8584 raft_consensus.cc:504] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.550099 10233 tablet_bootstrap.cc:492] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:31.550138 8584 leader_election.cc:288] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:31.550438 10233 tablet_bootstrap.cc:492] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:31.550544 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722306427b7b46b4b0942193c821d990" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:31.550601 10233 ts_tablet_manager.cc:1367] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.033s user 0.022s sys 0.006s
I20240426 09:49:31.550647 9669 raft_consensus.cc:3037] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.551061 10233 raft_consensus.cc:348] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.551199 10233 raft_consensus.cc:729] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.551340 10233 consensus_queue.cc:260] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.551347 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722306427b7b46b4b0942193c821d990" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:31.551487 9549 raft_consensus.cc:2454] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:31.551584 10233 ts_tablet_manager.cc:1398] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240426 09:49:31.551688 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722306427b7b46b4b0942193c821d990" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:31.551757 9669 raft_consensus.cc:2454] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:31.551805 9548 raft_consensus.cc:3037] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.552349 7208 leader_election.cc:302] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:31.552657 8584 raft_consensus.cc:2781] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:31.552745 8584 raft_consensus.cc:686] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:31.552779 9548 raft_consensus.cc:2454] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:31.552881 8584 consensus_queue.cc:237] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.553570 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-6dc287ff7ea640a38537c74e9ff156f7 received from {username='slave'} at 127.0.99.133:54839
I20240426 09:49:31.553682 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-6dc287ff7ea640a38537c74e9ff156f7 on tablet 6dc287ff7ea640a38537c74e9ff156f7 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:31.555898 6731 catalog_manager.cc:5484] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:31.566551 10268 ts_tablet_manager.cc:902] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:31.567332 10268 tablet_copy_client.cc:308] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:31.572146 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet c4bb42dad9a04eb78e0ee6453c8693b7 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:38611)
I20240426 09:49:31.572268 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet c4bb42dad9a04eb78e0ee6453c8693b7 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:38611: session id = 3b41d87e7bfb419f8aa7500100129b6c-c4bb42dad9a04eb78e0ee6453c8693b7
I20240426 09:49:31.574275 7033 tablet_copy_source_session.cc:205] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:31.574405 6882 consensus_queue.cc:237] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:31.575920 7142 raft_consensus.cc:1261] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.576073 10268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4bb42dad9a04eb78e0ee6453c8693b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.576318 9749 consensus_queue.cc:1035] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.578510 10268 tablet_copy_client.cc:783] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:31.578691 10268 tablet_copy_client.cc:647] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:31.579247 7275 raft_consensus.cc:1261] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:31.579270 6815 consensus_peers.cc:485] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c2fd1740d544f5d98741aecb64cdfc7. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.579790 9749 consensus_queue.cc:1035] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.580567 10268 tablet_copy_client.cc:523] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:31.581894 9749 raft_consensus.cc:2932] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.582298 10268 tablet_bootstrap.cc:492] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:31.582298 7275 raft_consensus.cc:2932] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.584055 7142 raft_consensus.cc:2932] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.587347 6739 catalog_manager.cc:5484] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.587910 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:31.596525 9975 tablet.cc:1722] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:31.596834 9975 tablet.cc:1620] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:31.597440 10232 tablet.cc:1722] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:31.597697 10232 tablet.cc:1620] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:31.600494 10231 tablet.cc:1722] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:31.600781 10231 tablet.cc:1620] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:31.601475 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=ff8716e2cc294ac0ac631a2db4575f63] alter complete (version 2)
I20240426 09:49:31.612792 10268 tablet_bootstrap.cc:492] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:31.613163 10268 tablet_bootstrap.cc:492] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:31.613389 10268 ts_tablet_manager.cc:1367] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.031s user 0.021s sys 0.007s
I20240426 09:49:31.613766 10268 raft_consensus.cc:348] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:31.613866 10268 raft_consensus.cc:729] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.614001 10268 consensus_queue.cc:260] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:31.614358 10268 ts_tablet_manager.cc:1398] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:31.614935 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-c4bb42dad9a04eb78e0ee6453c8693b7 received from {username='slave'} at 127.0.99.134:38611
I20240426 09:49:31.615053 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-c4bb42dad9a04eb78e0ee6453c8693b7 on tablet c4bb42dad9a04eb78e0ee6453c8693b7 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:31.618543 6882 consensus_queue.cc:237] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:31.620299 7011 raft_consensus.cc:1261] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.620309 7275 raft_consensus.cc:1261] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.620839 9200 consensus_queue.cc:1035] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.620988 10053 consensus_queue.cc:1035] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.622848 10303 ts_tablet_manager.cc:902] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:31.623564 10282 raft_consensus.cc:2932] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.623759 10303 tablet_copy_client.cc:308] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
W20240426 09:49:31.624011 6815 consensus_peers.cc:485] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e130e7733534aedb221d7772a2d29ce. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.625546 7011 raft_consensus.cc:2932] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.625865 7275 raft_consensus.cc:2932] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.628005 6739 catalog_manager.cc:5484] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.633752 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet f3b2b3e1d02b49269d19d284cab418da from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:41455)
I20240426 09:49:31.633878 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet f3b2b3e1d02b49269d19d284cab418da from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:41455: session id = 558c4ecd0ece460d9591aa7a3c799d19-f3b2b3e1d02b49269d19d284cab418da
I20240426 09:49:31.635293 7295 tablet_copy_source_session.cc:205] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:31.635347 10315 ts_tablet_manager.cc:902] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:31.636126 10315 tablet_copy_client.cc:308] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:31.636978 10303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3b2b3e1d02b49269d19d284cab418da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.638949 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet ce1e72dc91434c4ab63639fcc50da2ec from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:33529)
I20240426 09:49:31.639041 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet ce1e72dc91434c4ab63639fcc50da2ec from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:33529: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-ce1e72dc91434c4ab63639fcc50da2ec
I20240426 09:49:31.639453 10303 tablet_copy_client.cc:783] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:31.639622 10303 tablet_copy_client.cc:647] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:31.639796 7295 tablet_copy_source_session.cc:205] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:31.641484 10303 tablet_copy_client.cc:523] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:31.641520 10315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce1e72dc91434c4ab63639fcc50da2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.643256 10303 tablet_bootstrap.cc:492] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:31.643960 10315 tablet_copy_client.cc:783] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:31.644202 10315 tablet_copy_client.cc:647] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:31.645538 10315 tablet_copy_client.cc:523] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:31.647114 10315 tablet_bootstrap.cc:492] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:31.650791 10233 ts_tablet_manager.cc:902] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:31.651226 10233 tablet_copy_client.cc:308] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:31.656386 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet a8912e37d7e941a791dd77560e2b3b6b from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:40185)
I20240426 09:49:31.656498 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet a8912e37d7e941a791dd77560e2b3b6b from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:40185: session id = a692602e1a2e4794877e6fb92a7b9962-a8912e37d7e941a791dd77560e2b3b6b
I20240426 09:49:31.657403 7295 tablet_copy_source_session.cc:205] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:31.658332 10233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8912e37d7e941a791dd77560e2b3b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.660698 10233 tablet_copy_client.cc:783] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:31.660841 10233 tablet_copy_client.cc:647] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:31.664788 7274 consensus_queue.cc:237] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
W20240426 09:49:31.666481 7208 consensus_peers.cc:485] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a9c1e00452f4f32b31e8b336892df49. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.666589 7011 raft_consensus.cc:1261] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.667011 7142 raft_consensus.cc:1261] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.667269 10075 consensus_queue.cc:1035] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.667588 10075 consensus_queue.cc:1035] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.671068 8584 raft_consensus.cc:2932] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.672024 10233 tablet_copy_client.cc:523] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:31.672303 7011 raft_consensus.cc:2932] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.673593 10233 tablet_bootstrap.cc:492] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:31.674283 10303 tablet_bootstrap.cc:492] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:31.674714 10303 tablet_bootstrap.cc:492] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:31.674984 10303 ts_tablet_manager.cc:1367] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.032s user 0.031s sys 0.000s
I20240426 09:49:31.675310 7142 raft_consensus.cc:2932] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.675427 10303 raft_consensus.cc:348] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:31.675537 10303 raft_consensus.cc:729] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.675678 10303 consensus_queue.cc:260] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:31.676486 10303 ts_tablet_manager.cc:1398] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:31.676538 6731 catalog_manager.cc:5484] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.684979 10315 tablet_bootstrap.cc:492] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:31.685293 10315 tablet_bootstrap.cc:492] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:31.685458 10315 ts_tablet_manager.cc:1367] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.038s user 0.028s sys 0.000s
I20240426 09:49:31.685951 10345 ts_tablet_manager.cc:902] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:31.686375 10345 tablet_copy_client.cc:308] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:31.690824 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet aca60cffb8cf41e3b0d9b64436c306d7 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:49245)
I20240426 09:49:31.690944 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet aca60cffb8cf41e3b0d9b64436c306d7 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:49245: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-aca60cffb8cf41e3b0d9b64436c306d7
I20240426 09:49:31.692230 7164 tablet_copy_source_session.cc:205] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:31.693475 10345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca60cffb8cf41e3b0d9b64436c306d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.695667 10345 tablet_copy_client.cc:783] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:31.695818 10345 tablet_copy_client.cc:647] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:31.696002 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-f3b2b3e1d02b49269d19d284cab418da received from {username='slave'} at 127.0.99.136:41455
I20240426 09:49:31.696075 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-f3b2b3e1d02b49269d19d284cab418da on tablet f3b2b3e1d02b49269d19d284cab418da with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:31.698733 10315 raft_consensus.cc:348] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:31.698923 10315 raft_consensus.cc:729] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.699059 10315 consensus_queue.cc:260] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:31.699391 10315 ts_tablet_manager.cc:1398] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20240426 09:49:31.700553 10345 tablet_copy_client.cc:523] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:31.701392 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-ce1e72dc91434c4ab63639fcc50da2ec received from {username='slave'} at 127.0.99.135:33529
I20240426 09:49:31.701484 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-ce1e72dc91434c4ab63639fcc50da2ec on tablet ce1e72dc91434c4ab63639fcc50da2ec with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:31.702203 10345 tablet_bootstrap.cc:492] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:31.704828 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:31.707895 10233 tablet_bootstrap.cc:492] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:31.708204 10233 tablet_bootstrap.cc:492] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:31.708562 10233 ts_tablet_manager.cc:1367] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.000s
I20240426 09:49:31.709021 10233 raft_consensus.cc:348] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.709147 10233 raft_consensus.cc:729] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.709316 10233 consensus_queue.cc:260] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.710129 10233 ts_tablet_manager.cc:1398] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:31.711668 9975 tablet.cc:1722] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:31.713405 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-a8912e37d7e941a791dd77560e2b3b6b received from {username='slave'} at 127.0.99.133:40185
I20240426 09:49:31.713491 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-a8912e37d7e941a791dd77560e2b3b6b on tablet a8912e37d7e941a791dd77560e2b3b6b with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:31.714965 10232 tablet.cc:1722] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:31.717574 10231 tablet.cc:1722] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:31.718236 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=ff8716e2cc294ac0ac631a2db4575f63] alter complete (version 3)
I20240426 09:49:31.721179 10201 raft_consensus.cc:482] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:31.721266 10201 raft_consensus.cc:504] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.722381 10201 leader_election.cc:288] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:31.722930 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e856ff2c8ffb4bd4b44c694047c8a8fa" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:31.723062 7274 raft_consensus.cc:2454] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:31.723383 9616 leader_election.cc:302] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:31.723687 10201 raft_consensus.cc:2781] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:31.723737 10201 raft_consensus.cc:482] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:31.723767 10201 raft_consensus.cc:3037] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.724640 10201 raft_consensus.cc:504] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.724957 10201 leader_election.cc:288] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:31.725739 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e856ff2c8ffb4bd4b44c694047c8a8fa" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:31.725847 7274 raft_consensus.cc:3037] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.726855 7274 raft_consensus.cc:2454] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:31.727097 9616 leader_election.cc:302] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:31.727284 10201 raft_consensus.cc:2781] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:31.727391 10201 raft_consensus.cc:686] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:31.727527 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e856ff2c8ffb4bd4b44c694047c8a8fa" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:31.727639 9549 raft_consensus.cc:3037] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.727663 10201 consensus_queue.cc:237] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:31.728430 9549 raft_consensus.cc:2454] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:31.727527 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e856ff2c8ffb4bd4b44c694047c8a8fa" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:31.728926 9548 raft_consensus.cc:2362] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1. Re-sending same reply.
I20240426 09:49:31.730028 6732 catalog_manager.cc:5484] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:31.737633 10345 tablet_bootstrap.cc:492] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:31.737907 10345 tablet_bootstrap.cc:492] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:31.738055 10345 ts_tablet_manager.cc:1367] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.036s user 0.019s sys 0.003s
I20240426 09:49:31.738440 10345 raft_consensus.cc:348] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:31.738548 10345 raft_consensus.cc:729] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.738703 10345 consensus_queue.cc:260] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:31.739569 10345 ts_tablet_manager.cc:1398] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240426 09:49:31.740108 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-aca60cffb8cf41e3b0d9b64436c306d7 received from {username='slave'} at 127.0.99.135:49245
I20240426 09:49:31.740202 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-aca60cffb8cf41e3b0d9b64436c306d7 on tablet aca60cffb8cf41e3b0d9b64436c306d7 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:31.744758 7274 consensus_queue.cc:237] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.747025 7142 raft_consensus.cc:1261] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.747078 6882 raft_consensus.cc:1261] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.747459 8584 consensus_queue.cc:1035] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:49:31.747680 7209 consensus_peers.cc:485] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 295e6e95d86c47c88a09e47448c93d2e. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.747823 8584 consensus_queue.cc:1035] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.749956 8584 raft_consensus.cc:2932] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.750983 6880 raft_consensus.cc:2932] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.751853 7142 raft_consensus.cc:2932] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.752266 10303 ts_tablet_manager.cc:902] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:31.752732 10303 tablet_copy_client.cc:308] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:31.754045 6732 catalog_manager.cc:5484] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.761502 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet bcbf45071dd240c2ab6e75a9beac5b31 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:44679)
I20240426 09:49:31.761610 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet bcbf45071dd240c2ab6e75a9beac5b31 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:44679: session id = 558c4ecd0ece460d9591aa7a3c799d19-bcbf45071dd240c2ab6e75a9beac5b31
I20240426 09:49:31.762581 7033 tablet_copy_source_session.cc:205] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:31.763482 10303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbf45071dd240c2ab6e75a9beac5b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.765779 10303 tablet_copy_client.cc:783] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:31.765938 10303 tablet_copy_client.cc:647] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:31.767185 10303 tablet_copy_client.cc:523] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:31.768266 10303 tablet_bootstrap.cc:492] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:31.783210 7011 consensus_queue.cc:237] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:31.784777 7142 raft_consensus.cc:1261] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:31.784771 6946 consensus_peers.cc:485] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd077eafe82d4531a6850e6686a7e566. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.785161 7274 raft_consensus.cc:1261] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.785145 8988 consensus_queue.cc:1035] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.785549 10256 consensus_queue.cc:1035] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.787396 10256 raft_consensus.cc:2932] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.787832 7274 raft_consensus.cc:2932] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.789376 7142 raft_consensus.cc:2932] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.790979 6732 catalog_manager.cc:5484] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.798141 10303 tablet_bootstrap.cc:492] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:31.798429 10303 tablet_bootstrap.cc:492] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:31.798597 10268 ts_tablet_manager.cc:902] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:31.798609 10303 ts_tablet_manager.cc:1367] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.030s user 0.024s sys 0.000s
I20240426 09:49:31.799048 10268 tablet_copy_client.cc:308] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:31.799090 10303 raft_consensus.cc:348] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:31.799193 10303 raft_consensus.cc:729] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.799338 10303 consensus_queue.cc:260] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:31.799772 10303 ts_tablet_manager.cc:1398] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240426 09:49:31.800371 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-bcbf45071dd240c2ab6e75a9beac5b31 received from {username='slave'} at 127.0.99.136:44679
I20240426 09:49:31.800446 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-bcbf45071dd240c2ab6e75a9beac5b31 on tablet bcbf45071dd240c2ab6e75a9beac5b31 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:31.803750 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 0f67d94463ce46708246808da0b100e9 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:52457)
I20240426 09:49:31.803835 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 0f67d94463ce46708246808da0b100e9 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:52457: session id = 3b41d87e7bfb419f8aa7500100129b6c-0f67d94463ce46708246808da0b100e9
I20240426 09:49:31.804512 7164 tablet_copy_source_session.cc:205] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:31.805356 10268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f67d94463ce46708246808da0b100e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.807401 10268 tablet_copy_client.cc:783] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:31.807552 10268 tablet_copy_client.cc:647] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:31.808652 10268 tablet_copy_client.cc:523] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:31.809813 10268 tablet_bootstrap.cc:492] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:31.815060 7011 consensus_queue.cc:237] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:31.816561 7274 raft_consensus.cc:1261] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.816581 7142 raft_consensus.cc:1261] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:31.816803 6946 consensus_peers.cc:485] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11a10ff62d224cf49fcb71d11b029537. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.816956 10033 consensus_queue.cc:1035] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.817104 8988 consensus_queue.cc:1035] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.819247 8988 raft_consensus.cc:2932] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.819685 7274 raft_consensus.cc:2932] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.823170 6739 catalog_manager.cc:5484] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.823493 6731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:31.823592 6731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:31.824502 7142 raft_consensus.cc:2932] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.828367 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet f15176ddc6654756bc8042b3e66cacf6
I20240426 09:49:31.829018 9529 tablet_service.cc:1514] Processing DeleteTablet for tablet f15176ddc6654756bc8042b3e66cacf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:31.829080 9659 tablet_service.cc:1514] Processing DeleteTablet for tablet f15176ddc6654756bc8042b3e66cacf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:31.829519 10418 tablet_replica.cc:330] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:31.829850 10418 raft_consensus.cc:2229] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.829872 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet f15176ddc6654756bc8042b3e66cacf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:31.829950 10417 tablet_replica.cc:330] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:31.830045 10238 tablet_replica.cc:330] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:31.830072 10417 raft_consensus.cc:2229] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:31.830191 10238 raft_consensus.cc:2229] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.830317 10238 raft_consensus.cc:2258] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.830392 10417 raft_consensus.cc:2258] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.830988 10238 ts_tablet_manager.cc:1875] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.831056 10417 ts_tablet_manager.cc:1875] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.831820 10418 raft_consensus.cc:2258] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.833057 10418 ts_tablet_manager.cc:1875] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.833185 10238 ts_tablet_manager.cc:1888] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:31.833272 10238 log.cc:1196] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/f15176ddc6654756bc8042b3e66cacf6
I20240426 09:49:31.833527 10238 ts_tablet_manager.cc:1909] T f15176ddc6654756bc8042b3e66cacf6 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:31.833576 10417 ts_tablet_manager.cc:1888] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:31.833639 10417 log.cc:1196] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/f15176ddc6654756bc8042b3e66cacf6
I20240426 09:49:31.833886 10417 ts_tablet_manager.cc:1909] T f15176ddc6654756bc8042b3e66cacf6 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:31.834283 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet f15176ddc6654756bc8042b3e66cacf6 (table rebalancer_dynamic_table_new_name [id=ff8716e2cc294ac0ac631a2db4575f63]) successfully deleted
I20240426 09:49:31.834805 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet f15176ddc6654756bc8042b3e66cacf6 (table rebalancer_dynamic_table_new_name [id=ff8716e2cc294ac0ac631a2db4575f63]) successfully deleted
I20240426 09:49:31.835780 10268 tablet_bootstrap.cc:492] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:31.835988 10268 tablet_bootstrap.cc:492] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:31.836097 10268 ts_tablet_manager.cc:1367] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.000s
I20240426 09:49:31.836570 10418 ts_tablet_manager.cc:1888] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:31.836647 10418 log.cc:1196] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/f15176ddc6654756bc8042b3e66cacf6
I20240426 09:49:31.836566 10268 raft_consensus.cc:348] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:31.836680 10268 raft_consensus.cc:729] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.836907 10418 ts_tablet_manager.cc:1909] T f15176ddc6654756bc8042b3e66cacf6 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:31.836849 10268 consensus_queue.cc:260] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:31.837980 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet f15176ddc6654756bc8042b3e66cacf6 (table rebalancer_dynamic_table_new_name [id=ff8716e2cc294ac0ac631a2db4575f63]) successfully deleted
I20240426 09:49:31.838227 10268 ts_tablet_manager.cc:1398] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:31.838729 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-0f67d94463ce46708246808da0b100e9 received from {username='slave'} at 127.0.99.134:52457
I20240426 09:49:31.838840 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-0f67d94463ce46708246808da0b100e9 on tablet 0f67d94463ce46708246808da0b100e9 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:31.848867 7142 consensus_queue.cc:237] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:31.850324 6880 raft_consensus.cc:1261] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.850324 7011 raft_consensus.cc:1261] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:31.850469 7077 consensus_peers.cc:485] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d47b2dc234b342119e5dafea80caecbb. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.850708 10430 consensus_queue.cc:1035] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.850940 10112 consensus_queue.cc:1035] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.852849 10430 raft_consensus.cc:2932] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.853299 6880 raft_consensus.cc:2932] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.855806 6731 catalog_manager.cc:5484] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.857860 7011 raft_consensus.cc:2932] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.874577 7011 consensus_queue.cc:237] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:31.875993 7142 raft_consensus.cc:1261] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.876013 6880 raft_consensus.cc:1261] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.876379 10033 consensus_queue.cc:1035] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:49:31.876559 6946 consensus_peers.cc:485] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b85b6cfec5314d75aebe17670edf8d04. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.876570 10036 consensus_queue.cc:1035] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.879041 10036 raft_consensus.cc:2932] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.879439 7142 raft_consensus.cc:2932] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.880990 6880 raft_consensus.cc:2932] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.881863 6731 catalog_manager.cc:5484] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:31.901011 7274 consensus_queue.cc:237] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.902259 7142 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.902331 7011 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:31.902659 7209 consensus_peers.cc:485] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2d42c52b7a94e32ab4cdad78ce884ee. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.902719 10008 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.902895 10365 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.904635 10008 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.905040 7142 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.907959 6739 catalog_manager.cc:5484] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.906693 7011 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:31.909144 10033 consensus_queue.cc:1035] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.913079 9548 raft_consensus.cc:3037] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:31.914507 10033 consensus_queue.cc:1035] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.928889 7011 consensus_queue.cc:237] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:31.930362 7274 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.930816 7142 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.931212 10033 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.931584 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:31.933807 10033 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:49:31.934002 6946 consensus_peers.cc:485] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c94c820ab65420a8d3f885ccc9f8884. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.937332 9659 tablet_service.cc:1467] Processing CreateTablet for tablet a0d1aab8d2374ad2937c730ab10774dc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=14ab389461ea49b1872a19670ccc6ba1]), partition=
I20240426 09:49:31.937633 9659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d1aab8d2374ad2937c730ab10774dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.937624 9266 tablet_service.cc:1467] Processing CreateTablet for tablet a0d1aab8d2374ad2937c730ab10774dc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=14ab389461ea49b1872a19670ccc6ba1]), partition=
I20240426 09:49:31.937844 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d1aab8d2374ad2937c730ab10774dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.939945 7122 tablet_service.cc:1467] Processing CreateTablet for tablet a0d1aab8d2374ad2937c730ab10774dc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=14ab389461ea49b1872a19670ccc6ba1]), partition=
I20240426 09:49:31.940181 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0d1aab8d2374ad2937c730ab10774dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:31.940569 10490 tablet_bootstrap.cc:492] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:31.941514 10490 tablet_bootstrap.cc:654] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.942505 10160 tablet_bootstrap.cc:492] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:31.943096 10492 tablet_bootstrap.cc:492] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:31.943378 10160 tablet_bootstrap.cc:654] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.943935 10492 tablet_bootstrap.cc:654] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:31.944895 10492 tablet_bootstrap.cc:492] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:31.944975 10492 ts_tablet_manager.cc:1367] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:31.945365 10492 raft_consensus.cc:348] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.945472 10492 raft_consensus.cc:374] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.945502 10492 raft_consensus.cc:729] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.945614 10492 consensus_queue.cc:260] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.945902 10492 ts_tablet_manager.cc:1398] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:31.946477 10036 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.947425 10490 tablet_bootstrap.cc:492] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:31.947513 10490 ts_tablet_manager.cc:1367] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:31.947970 10490 raft_consensus.cc:348] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.948112 10490 raft_consensus.cc:374] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.948161 10490 raft_consensus.cc:729] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.948284 10490 consensus_queue.cc:260] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.948601 10490 ts_tablet_manager.cc:1398] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:31.948487 7142 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.948858 7274 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.951262 10160 tablet_bootstrap.cc:492] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:31.951345 10160 ts_tablet_manager.cc:1367] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240426 09:49:31.951751 10160 raft_consensus.cc:348] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.951846 10160 raft_consensus.cc:374] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:31.951875 10160 raft_consensus.cc:729] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:31.951985 10160 consensus_queue.cc:260] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.952268 10160 ts_tablet_manager.cc:1398] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:31.954941 6732 catalog_manager.cc:5484] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:31.975898 10365 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.978905 6880 consensus_queue.cc:237] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:31.980396 7274 raft_consensus.cc:1261] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.981114 7011 raft_consensus.cc:1261] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:31.981308 10008 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:31.981606 10053 consensus_queue.cc:1035] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.981961 10053 consensus_queue.cc:1035] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:31.984231 10053 raft_consensus.cc:2932] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.984647 7274 raft_consensus.cc:2932] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
W20240426 09:49:31.985958 6815 consensus_peers.cc:485] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 69179df98341493987f1d70ab24f62ef. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:31.986373 7011 raft_consensus.cc:2932] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.988070 6739 catalog_manager.cc:5484] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:31.988646 6731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240426 09:49:31.988755 6731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240426 09:49:31.991361 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 4 replicas of tablet 74c34398941848f5989acb2a1a88a82f
I20240426 09:49:31.991415 10233 ts_tablet_manager.cc:902] T 74c34398941848f5989acb2a1a88a82f P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:31.991746 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 69cb7b27400b4adc85071876764609f9
I20240426 09:49:31.991906 10233 tablet_copy_client.cc:308] T 74c34398941848f5989acb2a1a88a82f P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:31.992002 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 4 replicas of tablet c4bb42dad9a04eb78e0ee6453c8693b7
I20240426 09:49:31.992714 6861 tablet_service.cc:1514] Processing DeleteTablet for tablet 74c34398941848f5989acb2a1a88a82f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:31.992975 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 69cb7b27400b4adc85071876764609f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:31.993116 6860 tablet_service.cc:1514] Processing DeleteTablet for tablet c4bb42dad9a04eb78e0ee6453c8693b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:31.993261 9894 tablet_replica.cc:330] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:31.993389 9894 raft_consensus.cc:2229] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.993391 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 69cb7b27400b4adc85071876764609f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:31.993472 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet c4bb42dad9a04eb78e0ee6453c8693b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:31.993563 9894 raft_consensus.cc:2258] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.994149 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 74c34398941848f5989acb2a1a88a82f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:31.994236 10519 tablet_replica.cc:330] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:31.994309 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet c4bb42dad9a04eb78e0ee6453c8693b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:31.994151 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 69cb7b27400b4adc85071876764609f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:31.994679 10238 tablet_replica.cc:330] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:31.994807 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 74c34398941848f5989acb2a1a88a82f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:31.994836 10238 raft_consensus.cc:2229] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.994894 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet c4bb42dad9a04eb78e0ee6453c8693b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:31.995007 10238 raft_consensus.cc:2258] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.995077 10521 tablet_replica.cc:330] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:31.995218 10521 raft_consensus.cc:2229] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:31.995430 10521 raft_consensus.cc:2258] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.996237 9894 ts_tablet_manager.cc:1875] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.996438 10520 tablet_replica.cc:330] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:31.996731 10520 raft_consensus.cc:2229] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:31.996973 10520 raft_consensus.cc:2258] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:31.997996 10520 ts_tablet_manager.cc:1875] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.998019 10521 ts_tablet_manager.cc:1875] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.998466 9894 ts_tablet_manager.cc:1888] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:31.998528 9894 log.cc:1196] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/74c34398941848f5989acb2a1a88a82f
I20240426 09:49:31.998618 10196 raft_consensus.cc:482] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:31.998709 10196 raft_consensus.cc:504] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:31.999403 10238 ts_tablet_manager.cc:1875] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:31.999855 10196 leader_election.cc:288] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:31.999995 10519 raft_consensus.cc:2229] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:32.000346 10521 ts_tablet_manager.cc:1888] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.000689 10521 log.cc:1196] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/74c34398941848f5989acb2a1a88a82f
I20240426 09:49:32.000710 10519 raft_consensus.cc:2258] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.000970 10521 ts_tablet_manager.cc:1909] T 74c34398941848f5989acb2a1a88a82f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:32.001231 10519 ts_tablet_manager.cc:1875] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.001559 10238 ts_tablet_manager.cc:1888] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.001623 10238 log.cc:1196] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/74c34398941848f5989acb2a1a88a82f
I20240426 09:49:32.001612 10521 tablet_replica.cc:330] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:32.001726 10521 raft_consensus.cc:2229] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:32.001945 10521 raft_consensus.cc:2258] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.002537 10238 ts_tablet_manager.cc:1909] T 74c34398941848f5989acb2a1a88a82f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:32.002601 10521 ts_tablet_manager.cc:1875] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.002880 10520 ts_tablet_manager.cc:1888] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.003023 10520 log.cc:1196] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/c4bb42dad9a04eb78e0ee6453c8693b7
I20240426 09:49:32.003289 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 74c34398941848f5989acb2a1a88a82f (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.003346 10520 ts_tablet_manager.cc:1909] T c4bb42dad9a04eb78e0ee6453c8693b7 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:32.003490 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 74c34398941848f5989acb2a1a88a82f from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:57967)
W20240426 09:49:32.003597 7033 tablet_copy_service.cc:478] P 0a0bfe28bcc0435ebe9a54a9d6416899: Error handling TabletCopyService RPC request from {username='slave'} at 127.0.99.133:57967: Unable to find specified tablet: 74c34398941848f5989acb2a1a88a82f: Not found: Tablet not found: 74c34398941848f5989acb2a1a88a82f
I20240426 09:49:32.004024 10519 ts_tablet_manager.cc:1888] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:32.004105 10519 log.cc:1196] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/69cb7b27400b4adc85071876764609f9
I20240426 09:49:32.004385 10519 ts_tablet_manager.cc:1909] T 69cb7b27400b4adc85071876764609f9 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:32.004660 10521 ts_tablet_manager.cc:1888] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.004813 10521 log.cc:1196] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/c4bb42dad9a04eb78e0ee6453c8693b7
I20240426 09:49:32.005335 10521 ts_tablet_manager.cc:1909] T c4bb42dad9a04eb78e0ee6453c8693b7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:32.005662 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 69cb7b27400b4adc85071876764609f9 (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.005826 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet c4bb42dad9a04eb78e0ee6453c8693b7 (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.005995 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet c4bb42dad9a04eb78e0ee6453c8693b7 (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.006489 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 74c34398941848f5989acb2a1a88a82f (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.006961 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 74c34398941848f5989acb2a1a88a82f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:31 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:32.007412 9894 ts_tablet_manager.cc:1909] T 74c34398941848f5989acb2a1a88a82f P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:32.007974 9894 tablet_replica.cc:330] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:32.008105 9894 raft_consensus.cc:2229] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.008138 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 74c34398941848f5989acb2a1a88a82f (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.008234 9894 raft_consensus.cc:2258] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.008484 10238 tablet_replica.cc:330] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:32.008617 10238 raft_consensus.cc:2229] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
W20240426 09:49:32.008600 6725 catalog_manager.cc:4797] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): delete failed for tablet 74c34398941848f5989acb2a1a88a82f because the tablet was not found. No further retry: Not found: Tablet not found: 74c34398941848f5989acb2a1a88a82f
I20240426 09:49:32.008760 10238 raft_consensus.cc:2258] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.009279 9894 ts_tablet_manager.cc:1875] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.009961 10238 ts_tablet_manager.cc:1875] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.011700 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d1aab8d2374ad2937c730ab10774dc" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:32.011854 9669 raft_consensus.cc:2454] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:32.013008 9219 leader_election.cc:302] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:32.013324 10196 raft_consensus.cc:2781] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:32.013391 10196 raft_consensus.cc:482] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:32.013429 10196 raft_consensus.cc:3037] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.014271 10196 raft_consensus.cc:504] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:32.014333 9894 ts_tablet_manager.cc:1888] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:32.014401 9894 log.cc:1196] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/69cb7b27400b4adc85071876764609f9
I20240426 09:49:32.014573 10196 leader_election.cc:288] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:32.014901 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d1aab8d2374ad2937c730ab10774dc" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:32.014920 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d1aab8d2374ad2937c730ab10774dc" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:32.015012 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0d1aab8d2374ad2937c730ab10774dc" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:32.015051 7142 raft_consensus.cc:2454] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:32.015108 9669 raft_consensus.cc:3037] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.015509 10238 ts_tablet_manager.cc:1888] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.015560 10238 log.cc:1196] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/c4bb42dad9a04eb78e0ee6453c8693b7
I20240426 09:49:32.015745 10238 ts_tablet_manager.cc:1909] T c4bb42dad9a04eb78e0ee6453c8693b7 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:32.014672 9894 ts_tablet_manager.cc:1909] T 69cb7b27400b4adc85071876764609f9 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:32.016036 9669 raft_consensus.cc:2454] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:32.016320 9219 leader_election.cc:302] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters: 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:32.016395 9894 tablet_replica.cc:330] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:32.016515 9894 raft_consensus.cc:2229] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.016594 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 69cb7b27400b4adc85071876764609f9 (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.016646 9894 raft_consensus.cc:2258] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.016779 10196 raft_consensus.cc:2781] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:32.016938 10196 raft_consensus.cc:686] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:32.017180 10196 consensus_queue.cc:237] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:32.017333 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet c4bb42dad9a04eb78e0ee6453c8693b7 (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.017355 9894 ts_tablet_manager.cc:1875] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.017534 10238 tablet_replica.cc:330] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:32.017783 10238 raft_consensus.cc:2229] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.017915 10238 raft_consensus.cc:2258] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.018402 10238 ts_tablet_manager.cc:1875] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.019351 9894 ts_tablet_manager.cc:1888] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.019428 9894 log.cc:1196] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/c4bb42dad9a04eb78e0ee6453c8693b7
I20240426 09:49:32.019393 6731 catalog_manager.cc:5484] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:32.019704 9894 ts_tablet_manager.cc:1909] T c4bb42dad9a04eb78e0ee6453c8693b7 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:32.020969 10238 ts_tablet_manager.cc:1888] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:32.021046 10238 log.cc:1196] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/69cb7b27400b4adc85071876764609f9
I20240426 09:49:32.021029 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet c4bb42dad9a04eb78e0ee6453c8693b7 (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.021517 10238 ts_tablet_manager.cc:1909] T 69cb7b27400b4adc85071876764609f9 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:32.022179 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 69cb7b27400b4adc85071876764609f9 (table rebalance_test_table_6 [id=5a9462a663cc4573b497b6fb37b99612]) successfully deleted
I20240426 09:49:32.042671 9285 raft_consensus.cc:1203] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.055116 9285 raft_consensus.cc:1203] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.060499 9548 raft_consensus.cc:1203] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.091660 6860 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:32.093014 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:32.093158 7255 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:32.095181 7122 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:32.095654 6991 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:32.120076 9659 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:32.121289 9529 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:32.131606 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:32.134037 9397 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:32.136883 7143 raft_consensus.cc:3037] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.136951 9669 raft_consensus.cc:1261] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:32.137574 10535 consensus_queue.cc:1035] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:32.137995 7143 raft_consensus.cc:1261] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:32.138382 10535 consensus_queue.cc:1035] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:32.141417 6739 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet a0d1aab8d2374ad2937c730ab10774dc (table rebalancer_dynamic_table [id=14ab389461ea49b1872a19670ccc6ba1]). Expected version 1 got 0
I20240426 09:49:32.142796 10582 tablet.cc:1722] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:32.143241 10582 tablet.cc:1620] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:32.144522 10583 tablet.cc:1722] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:32.144961 10583 tablet.cc:1620] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:32.145272 6739 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet a0d1aab8d2374ad2937c730ab10774dc (table rebalancer_dynamic_table [id=14ab389461ea49b1872a19670ccc6ba1]). Expected version 1 got 0
I20240426 09:49:32.152429 10233 ts_tablet_manager.cc:902] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:32.152968 10233 tablet_copy_client.cc:308] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:32.153483 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 295e6e95d86c47c88a09e47448c93d2e from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:40185)
I20240426 09:49:32.153641 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 295e6e95d86c47c88a09e47448c93d2e from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:40185: session id = a692602e1a2e4794877e6fb92a7b9962-295e6e95d86c47c88a09e47448c93d2e
I20240426 09:49:32.154646 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=14ab389461ea49b1872a19670ccc6ba1] alter complete (version 1)
I20240426 09:49:32.155627 7295 tablet_copy_source_session.cc:205] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.156635 10233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 295e6e95d86c47c88a09e47448c93d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.158537 10233 tablet_copy_client.cc:783] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.158697 10233 tablet_copy_client.cc:647] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.162448 10201 consensus_queue.cc:1035] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:32.163197 10233 tablet_copy_client.cc:523] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.164547 10233 tablet_bootstrap.cc:492] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:32.170852 10268 ts_tablet_manager.cc:902] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:32.171288 10268 tablet_copy_client.cc:308] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:32.176263 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 7e130e7733534aedb221d7772a2d29ce from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:40097)
I20240426 09:49:32.176360 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 7e130e7733534aedb221d7772a2d29ce from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:40097: session id = 3b41d87e7bfb419f8aa7500100129b6c-7e130e7733534aedb221d7772a2d29ce
I20240426 09:49:32.177229 6902 tablet_copy_source_session.cc:205] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.178622 10268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e130e7733534aedb221d7772a2d29ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.178903 10201 consensus_queue.cc:1035] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:32.182739 10268 tablet_copy_client.cc:783] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.182926 10268 tablet_copy_client.cc:647] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.187616 10268 tablet_copy_client.cc:523] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.188730 10268 tablet_bootstrap.cc:492] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:32.197968 10303 ts_tablet_manager.cc:902] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:32.198428 10303 tablet_copy_client.cc:308] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:32.202706 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 1c2fd1740d544f5d98741aecb64cdfc7 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:56221)
I20240426 09:49:32.202829 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 1c2fd1740d544f5d98741aecb64cdfc7 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:56221: session id = 558c4ecd0ece460d9591aa7a3c799d19-1c2fd1740d544f5d98741aecb64cdfc7
I20240426 09:49:32.203719 6902 tablet_copy_source_session.cc:205] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.204519 10303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c2fd1740d544f5d98741aecb64cdfc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.206633 10303 tablet_copy_client.cc:783] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.206830 10233 tablet_bootstrap.cc:492] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.207134 10233 tablet_bootstrap.cc:492] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:32.207311 10233 ts_tablet_manager.cc:1367] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.043s user 0.026s sys 0.001s
I20240426 09:49:32.207769 10233 raft_consensus.cc:348] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:32.207895 10233 raft_consensus.cc:729] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.208052 10233 consensus_queue.cc:260] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:32.209323 10303 tablet_copy_client.cc:647] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.210465 10233 ts_tablet_manager.cc:1398] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:32.210664 10231 tablet.cc:1722] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:32.211337 10231 tablet.cc:1620] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:32.212227 10303 tablet_copy_client.cc:523] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.212411 6732 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet a0d1aab8d2374ad2937c730ab10774dc (table rebalancer_dynamic_table [id=14ab389461ea49b1872a19670ccc6ba1]). Expected version 1 got 0
I20240426 09:49:32.213454 10303 tablet_bootstrap.cc:492] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:32.214015 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-295e6e95d86c47c88a09e47448c93d2e received from {username='slave'} at 127.0.99.133:40185
I20240426 09:49:32.214123 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-295e6e95d86c47c88a09e47448c93d2e on tablet 295e6e95d86c47c88a09e47448c93d2e with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:32.222005 10268 tablet_bootstrap.cc:492] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.222324 10268 tablet_bootstrap.cc:492] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:32.222513 10268 ts_tablet_manager.cc:1367] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.034s user 0.026s sys 0.000s
I20240426 09:49:32.223088 10268 raft_consensus.cc:348] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.223315 10268 raft_consensus.cc:729] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.223524 10268 consensus_queue.cc:260] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.223881 10268 ts_tablet_manager.cc:1398] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:32.224853 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-7e130e7733534aedb221d7772a2d29ce received from {username='slave'} at 127.0.99.134:40097
I20240426 09:49:32.224936 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-7e130e7733534aedb221d7772a2d29ce on tablet 7e130e7733534aedb221d7772a2d29ce with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:32.224961 10345 ts_tablet_manager.cc:902] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:32.225355 10345 tablet_copy_client.cc:308] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:32.225849 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 9a9c1e00452f4f32b31e8b336892df49 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:33529)
I20240426 09:49:32.225991 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 9a9c1e00452f4f32b31e8b336892df49 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:33529: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-9a9c1e00452f4f32b31e8b336892df49
I20240426 09:49:32.227140 7295 tablet_copy_source_session.cc:205] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.228139 9417 raft_consensus.cc:1203] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.228700 10345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9c1e00452f4f32b31e8b336892df49. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.230515 10345 tablet_copy_client.cc:783] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.230667 10345 tablet_copy_client.cc:647] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.233173 10345 tablet_copy_client.cc:523] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.234469 10345 tablet_bootstrap.cc:492] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:32.237926 10303 tablet_bootstrap.cc:492] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.238202 10303 tablet_bootstrap.cc:492] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:32.238359 10303 ts_tablet_manager.cc:1367] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.025s user 0.017s sys 0.002s
I20240426 09:49:32.238917 10303 raft_consensus.cc:348] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.239085 10303 raft_consensus.cc:729] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.239264 10303 consensus_queue.cc:260] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.239528 10303 ts_tablet_manager.cc:1398] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240426 09:49:32.239976 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-1c2fd1740d544f5d98741aecb64cdfc7 received from {username='slave'} at 127.0.99.136:56221
I20240426 09:49:32.240058 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-1c2fd1740d544f5d98741aecb64cdfc7 on tablet 1c2fd1740d544f5d98741aecb64cdfc7 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:32.242657 9669 raft_consensus.cc:1203] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.255280 10345 tablet_bootstrap.cc:492] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.255563 10345 tablet_bootstrap.cc:492] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:32.255724 10345 ts_tablet_manager.cc:1367] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.003s
I20240426 09:49:32.256152 10345 raft_consensus.cc:348] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:32.256263 10345 raft_consensus.cc:729] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.256397 10345 consensus_queue.cc:260] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:32.256677 10345 ts_tablet_manager.cc:1398] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:32.257141 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-9a9c1e00452f4f32b31e8b336892df49 received from {username='slave'} at 127.0.99.135:33529
I20240426 09:49:32.257225 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-9a9c1e00452f4f32b31e8b336892df49 on tablet 9a9c1e00452f4f32b31e8b336892df49 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:32.260694 9548 raft_consensus.cc:1203] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.261845 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:32.266942 10582 tablet.cc:1722] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:32.267263 10582 tablet.cc:1620] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:32.267364 10231 tablet.cc:1722] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:32.267669 10231 tablet.cc:1620] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:32.270344 10583 tablet.cc:1722] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:32.270921 10583 tablet.cc:1620] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:32.273588 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=14ab389461ea49b1872a19670ccc6ba1] alter complete (version 2)
I20240426 09:49:32.276212 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:32.282922 9266 tablet_service.cc:1467] Processing CreateTablet for tablet ce43c4c9b2994813ad7ad38f2e1b1040 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:32.283226 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce43c4c9b2994813ad7ad38f2e1b1040. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.284032 9659 tablet_service.cc:1467] Processing CreateTablet for tablet ce43c4c9b2994813ad7ad38f2e1b1040 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:32.284327 9659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce43c4c9b2994813ad7ad38f2e1b1040. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.284317 9397 tablet_service.cc:1467] Processing CreateTablet for tablet bbdcd6105a2647738257638e1e1096c3 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:32.284587 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdcd6105a2647738257638e1e1096c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.285332 10490 tablet_bootstrap.cc:492] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:32.286432 10490 tablet_bootstrap.cc:654] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.286474 10648 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:32.286526 9660 tablet_service.cc:1467] Processing CreateTablet for tablet bbdcd6105a2647738257638e1e1096c3 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:32.286641 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 5b5e68d9b3ce49d3aa398e85e139ddbf (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:32.286760 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdcd6105a2647738257638e1e1096c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.286916 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5e68d9b3ce49d3aa398e85e139ddbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.287073 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 5b5e68d9b3ce49d3aa398e85e139ddbf (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:32.287456 10160 tablet_bootstrap.cc:492] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:32.289638 10648 tablet_bootstrap.cc:654] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.290174 6860 tablet_service.cc:1467] Processing CreateTablet for tablet ce43c4c9b2994813ad7ad38f2e1b1040 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:32.290406 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 5b5e68d9b3ce49d3aa398e85e139ddbf (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:32.290479 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce43c4c9b2994813ad7ad38f2e1b1040. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.290614 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5e68d9b3ce49d3aa398e85e139ddbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.290720 10648 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:32.290814 10648 ts_tablet_manager.cc:1367] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:49:32.291304 10648 raft_consensus.cc:348] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.291383 10648 raft_consensus.cc:374] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.291402 10648 raft_consensus.cc:729] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.291517 10648 consensus_queue.cc:260] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.291788 10648 ts_tablet_manager.cc:1398] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:32.291990 10648 tablet_bootstrap.cc:492] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:32.292826 10648 tablet_bootstrap.cc:654] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.292964 10651 tablet_bootstrap.cc:492] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:32.293988 10651 tablet_bootstrap.cc:654] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.295892 9266 tablet_service.cc:1467] Processing CreateTablet for tablet bbdcd6105a2647738257638e1e1096c3 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:32.296128 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdcd6105a2647738257638e1e1096c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.296895 10160 tablet_bootstrap.cc:654] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.298127 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5e68d9b3ce49d3aa398e85e139ddbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.299003 10490 tablet_bootstrap.cc:492] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:32.299060 10648 tablet_bootstrap.cc:492] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:32.299103 10490 ts_tablet_manager.cc:1367] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240426 09:49:32.299135 10648 ts_tablet_manager.cc:1367] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240426 09:49:32.299398 10160 tablet_bootstrap.cc:492] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:32.299489 10160 ts_tablet_manager.cc:1367] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240426 09:49:32.299542 10490 raft_consensus.cc:348] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.299573 10648 raft_consensus.cc:348] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.299647 10490 raft_consensus.cc:374] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.299669 10648 raft_consensus.cc:374] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.299681 10490 raft_consensus.cc:729] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.299696 10648 raft_consensus.cc:729] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.299813 10490 consensus_queue.cc:260] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.299826 10648 consensus_queue.cc:260] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.299933 10160 raft_consensus.cc:348] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.300022 10160 raft_consensus.cc:374] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.300050 10160 raft_consensus.cc:729] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.300081 10490 ts_tablet_manager.cc:1398] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:32.300163 10160 consensus_queue.cc:260] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.300256 10490 tablet_bootstrap.cc:492] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:32.300453 10160 ts_tablet_manager.cc:1398] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:32.300566 10648 ts_tablet_manager.cc:1398] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:32.301153 10490 tablet_bootstrap.cc:654] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.301548 10160 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:32.302544 10160 tablet_bootstrap.cc:654] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.303515 10490 tablet_bootstrap.cc:492] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:32.303593 10490 ts_tablet_manager.cc:1367] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:32.303946 10651 tablet_bootstrap.cc:492] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:32.304056 10651 ts_tablet_manager.cc:1367] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:32.303982 10490 raft_consensus.cc:348] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.304119 10490 raft_consensus.cc:374] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.304168 10490 raft_consensus.cc:729] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.304267 10490 consensus_queue.cc:260] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.304526 10490 ts_tablet_manager.cc:1398] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:32.304674 10490 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:32.304782 10651 raft_consensus.cc:348] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.304878 10651 raft_consensus.cc:374] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.304911 10651 raft_consensus.cc:729] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.305022 10651 consensus_queue.cc:260] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.305301 10651 ts_tablet_manager.cc:1398] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:32.305514 10490 tablet_bootstrap.cc:654] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.309623 10490 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:32.309738 10490 ts_tablet_manager.cc:1367] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240426 09:49:32.309732 9669 raft_consensus.cc:1203] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.310164 10490 raft_consensus.cc:348] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.310346 10490 raft_consensus.cc:374] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.310436 10490 raft_consensus.cc:729] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.310628 10490 consensus_queue.cc:260] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.311373 10490 ts_tablet_manager.cc:1398] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:32.323415 10160 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:32.323614 10160 ts_tablet_manager.cc:1367] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.001s
I20240426 09:49:32.324308 10160 raft_consensus.cc:348] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.324442 10160 raft_consensus.cc:374] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.324492 10160 raft_consensus.cc:729] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.324636 10160 consensus_queue.cc:260] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.325501 10160 ts_tablet_manager.cc:1398] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:32.325683 10160 tablet_bootstrap.cc:492] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:32.326833 10160 tablet_bootstrap.cc:654] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.328550 10160 tablet_bootstrap.cc:492] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:32.328666 10160 ts_tablet_manager.cc:1367] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:32.329121 10160 raft_consensus.cc:348] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.329274 10160 raft_consensus.cc:374] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.329393 10160 raft_consensus.cc:729] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.329515 10160 consensus_queue.cc:260] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.329813 10160 ts_tablet_manager.cc:1398] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:32.364612 10268 ts_tablet_manager.cc:902] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:32.365228 10268 tablet_copy_client.cc:308] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:32.365689 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 11a10ff62d224cf49fcb71d11b029537 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:38611)
I20240426 09:49:32.365784 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 11a10ff62d224cf49fcb71d11b029537 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:38611: session id = 3b41d87e7bfb419f8aa7500100129b6c-11a10ff62d224cf49fcb71d11b029537
I20240426 09:49:32.366648 7033 tablet_copy_source_session.cc:205] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.367507 10268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11a10ff62d224cf49fcb71d11b029537. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.369218 10268 tablet_copy_client.cc:783] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.369385 10268 tablet_copy_client.cc:647] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.370507 10268 tablet_copy_client.cc:523] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.371683 10268 tablet_bootstrap.cc:492] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:32.375780 10303 ts_tablet_manager.cc:902] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:32.376263 10303 tablet_copy_client.cc:308] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:32.376698 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet bd077eafe82d4531a6850e6686a7e566 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:44679)
I20240426 09:49:32.376791 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet bd077eafe82d4531a6850e6686a7e566 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:44679: session id = 558c4ecd0ece460d9591aa7a3c799d19-bd077eafe82d4531a6850e6686a7e566
I20240426 09:49:32.377671 7033 tablet_copy_source_session.cc:205] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.378479 10303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd077eafe82d4531a6850e6686a7e566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.378618 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:32.383190 10303 tablet_copy_client.cc:783] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.383352 10303 tablet_copy_client.cc:647] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.385372 10582 tablet.cc:1722] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:32.387832 10583 tablet.cc:1722] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:32.391224 10231 tablet.cc:1722] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:32.392812 10303 tablet_copy_client.cc:523] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.394578 10303 tablet_bootstrap.cc:492] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:32.395299 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=14ab389461ea49b1872a19670ccc6ba1] alter complete (version 3)
I20240426 09:49:32.399696 10268 tablet_bootstrap.cc:492] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.400007 10268 tablet_bootstrap.cc:492] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:32.400017 10671 ts_tablet_manager.cc:902] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:32.400182 10268 ts_tablet_manager.cc:1367] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.000s
I20240426 09:49:32.400514 10671 tablet_copy_client.cc:308] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:32.400751 10268 raft_consensus.cc:348] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.400892 10268 raft_consensus.cc:729] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.401021 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet b85b6cfec5314d75aebe17670edf8d04 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:44679)
I20240426 09:49:32.401075 10268 consensus_queue.cc:260] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.401126 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet b85b6cfec5314d75aebe17670edf8d04 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:44679: session id = 558c4ecd0ece460d9591aa7a3c799d19-b85b6cfec5314d75aebe17670edf8d04
I20240426 09:49:32.401419 10268 ts_tablet_manager.cc:1398] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:32.402123 7033 tablet_copy_source_session.cc:205] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.402141 7032 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-11a10ff62d224cf49fcb71d11b029537 received from {username='slave'} at 127.0.99.134:38611
I20240426 09:49:32.402218 7032 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-11a10ff62d224cf49fcb71d11b029537 on tablet 11a10ff62d224cf49fcb71d11b029537 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:32.402940 10671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b85b6cfec5314d75aebe17670edf8d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.414705 10303 tablet_bootstrap.cc:492] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.427026 10303 tablet_bootstrap.cc:492] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:32.427249 10303 ts_tablet_manager.cc:1367] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.033s user 0.011s sys 0.007s
I20240426 09:49:32.427661 10303 raft_consensus.cc:348] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.427762 10303 raft_consensus.cc:729] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.427902 10303 consensus_queue.cc:260] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.428198 10303 ts_tablet_manager.cc:1398] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:32.428644 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-bd077eafe82d4531a6850e6686a7e566 received from {username='slave'} at 127.0.99.136:44679
I20240426 09:49:32.428731 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-bd077eafe82d4531a6850e6686a7e566 on tablet bd077eafe82d4531a6850e6686a7e566 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:32.428722 10671 tablet_copy_client.cc:783] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.428871 10671 tablet_copy_client.cc:647] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.429963 10671 tablet_copy_client.cc:523] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.431242 10671 tablet_bootstrap.cc:492] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:32.446056 10345 ts_tablet_manager.cc:902] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:32.446569 10345 tablet_copy_client.cc:308] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:32.447062 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet d47b2dc234b342119e5dafea80caecbb from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:49245)
I20240426 09:49:32.447167 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet d47b2dc234b342119e5dafea80caecbb from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:49245: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-d47b2dc234b342119e5dafea80caecbb
I20240426 09:49:32.448036 7164 tablet_copy_source_session.cc:205] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.448906 10345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47b2dc234b342119e5dafea80caecbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.451331 10671 tablet_bootstrap.cc:492] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.451337 10345 tablet_copy_client.cc:783] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.451504 10345 tablet_copy_client.cc:647] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.451615 10268 ts_tablet_manager.cc:902] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:32.451642 10671 tablet_bootstrap.cc:492] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:32.451807 10671 ts_tablet_manager.cc:1367] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.021s user 0.017s sys 0.003s
I20240426 09:49:32.452095 10268 tablet_copy_client.cc:308] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:32.452342 10671 raft_consensus.cc:348] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.452479 10671 raft_consensus.cc:729] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.452527 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 9c94c820ab65420a8d3f885ccc9f8884 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:38611)
I20240426 09:49:32.452625 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 9c94c820ab65420a8d3f885ccc9f8884 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:38611: session id = 3b41d87e7bfb419f8aa7500100129b6c-9c94c820ab65420a8d3f885ccc9f8884
I20240426 09:49:32.452644 10671 consensus_queue.cc:260] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.452916 10671 ts_tablet_manager.cc:1398] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:32.453318 10345 tablet_copy_client.cc:523] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.453346 7032 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-b85b6cfec5314d75aebe17670edf8d04 received from {username='slave'} at 127.0.99.136:44679
I20240426 09:49:32.453413 7032 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-b85b6cfec5314d75aebe17670edf8d04 on tablet b85b6cfec5314d75aebe17670edf8d04 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:32.453619 7033 tablet_copy_source_session.cc:205] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.454551 10345 tablet_bootstrap.cc:492] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:32.454576 10268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c94c820ab65420a8d3f885ccc9f8884. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.456408 10268 tablet_copy_client.cc:783] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.456547 10268 tablet_copy_client.cc:647] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.457571 10268 tablet_copy_client.cc:523] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.458726 10268 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:32.480692 10345 tablet_bootstrap.cc:492] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.481024 10345 tablet_bootstrap.cc:492] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:32.481202 10345 ts_tablet_manager.cc:1367] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.027s user 0.019s sys 0.006s
I20240426 09:49:32.481643 10345 raft_consensus.cc:348] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:32.481774 10345 raft_consensus.cc:729] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.481927 10345 consensus_queue.cc:260] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:32.482196 10345 ts_tablet_manager.cc:1398] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:32.482743 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-d47b2dc234b342119e5dafea80caecbb received from {username='slave'} at 127.0.99.135:49245
I20240426 09:49:32.482875 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-d47b2dc234b342119e5dafea80caecbb on tablet d47b2dc234b342119e5dafea80caecbb with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:32.484555 10268 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.484810 10268 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:32.484987 10268 ts_tablet_manager.cc:1367] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.026s user 0.026s sys 0.000s
I20240426 09:49:32.485488 10268 raft_consensus.cc:348] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.485622 10268 raft_consensus.cc:729] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.485762 10268 consensus_queue.cc:260] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.486029 10268 ts_tablet_manager.cc:1398] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:32.486611 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-9c94c820ab65420a8d3f885ccc9f8884 received from {username='slave'} at 127.0.99.134:38611
I20240426 09:49:32.486686 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-9c94c820ab65420a8d3f885ccc9f8884 on tablet 9c94c820ab65420a8d3f885ccc9f8884 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:32.497392 6737 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:32.497545 6737 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:32.498245 10196 raft_consensus.cc:482] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:32.498363 10196 raft_consensus.cc:504] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.498868 10196 leader_election.cc:288] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:32.499914 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5e68d9b3ce49d3aa398e85e139ddbf" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:32.500080 9672 raft_consensus.cc:2454] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:32.500983 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet a0d1aab8d2374ad2937c730ab10774dc
I20240426 09:49:32.501332 9219 leader_election.cc:302] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:32.501559 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d1aab8d2374ad2937c730ab10774dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:32.501717 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d1aab8d2374ad2937c730ab10774dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:32.501969 10196 raft_consensus.cc:2781] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:32.502035 10196 raft_consensus.cc:482] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:32.502084 10196 raft_consensus.cc:3037] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.502137 10519 tablet_replica.cc:330] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:32.502137 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet a0d1aab8d2374ad2937c730ab10774dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:32.502282 10519 raft_consensus.cc:2229] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.502413 10519 raft_consensus.cc:2258] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.502609 10528 tablet_replica.cc:330] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:32.502734 10528 raft_consensus.cc:2229] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:32.503039 10519 ts_tablet_manager.cc:1875] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.503055 10528 raft_consensus.cc:2258] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.503131 10196 raft_consensus.cc:504] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.503481 10196 leader_election.cc:288] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:32.503540 10528 ts_tablet_manager.cc:1875] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.503901 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5e68d9b3ce49d3aa398e85e139ddbf" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:32.504009 9672 raft_consensus.cc:3037] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.504079 10687 tablet_replica.cc:330] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:32.504205 10687 raft_consensus.cc:2229] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.504325 10687 raft_consensus.cc:2258] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.504982 9672 raft_consensus.cc:2454] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:32.505146 10519 ts_tablet_manager.cc:1888] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.505206 10519 log.cc:1196] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/a0d1aab8d2374ad2937c730ab10774dc
I20240426 09:49:32.505368 10528 ts_tablet_manager.cc:1888] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.505422 10519 ts_tablet_manager.cc:1909] T a0d1aab8d2374ad2937c730ab10774dc P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:32.505430 10528 log.cc:1196] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/a0d1aab8d2374ad2937c730ab10774dc
I20240426 09:49:32.506227 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet a0d1aab8d2374ad2937c730ab10774dc (table rebalancer_dynamic_table_new_name [id=14ab389461ea49b1872a19670ccc6ba1]) successfully deleted
I20240426 09:49:32.507674 10528 ts_tablet_manager.cc:1909] T a0d1aab8d2374ad2937c730ab10774dc P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:32.508589 10687 ts_tablet_manager.cc:1875] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.508729 9219 leader_election.cc:302] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:32.508716 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet a0d1aab8d2374ad2937c730ab10774dc (table rebalancer_dynamic_table_new_name [id=14ab389461ea49b1872a19670ccc6ba1]) successfully deleted
I20240426 09:49:32.509233 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5e68d9b3ce49d3aa398e85e139ddbf" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:32.509362 9416 raft_consensus.cc:2454] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:32.509445 6737 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240426 09:49:32.509562 6737 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240426 09:49:32.509557 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b5e68d9b3ce49d3aa398e85e139ddbf" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:32.509650 9415 raft_consensus.cc:3037] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.510648 10196 raft_consensus.cc:2781] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:32.510654 9415 raft_consensus.cc:2454] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:32.510865 10687 ts_tablet_manager.cc:1888] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.510983 10687 log.cc:1196] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/a0d1aab8d2374ad2937c730ab10774dc
I20240426 09:49:32.510742 10196 raft_consensus.cc:686] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:32.511314 10687 ts_tablet_manager.cc:1909] T a0d1aab8d2374ad2937c730ab10774dc P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:32.511405 10196 consensus_queue.cc:237] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:32.511988 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet a0d1aab8d2374ad2937c730ab10774dc (table rebalancer_dynamic_table_new_name [id=14ab389461ea49b1872a19670ccc6ba1]) successfully deleted
I20240426 09:49:32.512964 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 4 replicas of tablet d47b2dc234b342119e5dafea80caecbb
I20240426 09:49:32.513350 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 4 replicas of tablet bcbf45071dd240c2ab6e75a9beac5b31
I20240426 09:49:32.513653 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 011cfc9512c54fe689ca5ba948e57fd8
I20240426 09:49:32.513700 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet d47b2dc234b342119e5dafea80caecbb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:32.514734 6732 catalog_manager.cc:5484] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:32.514918 10689 tablet_replica.cc:330] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:32.515065 10689 raft_consensus.cc:2229] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.515228 10689 raft_consensus.cc:2258] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.515425 6860 tablet_service.cc:1514] Processing DeleteTablet for tablet d47b2dc234b342119e5dafea80caecbb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:32.515590 9894 tablet_replica.cc:330] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:32.515746 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbf45071dd240c2ab6e75a9beac5b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:32.515818 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbf45071dd240c2ab6e75a9beac5b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:32.515875 6861 tablet_service.cc:1514] Processing DeleteTablet for tablet 011cfc9512c54fe689ca5ba948e57fd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:32.515928 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet d47b2dc234b342119e5dafea80caecbb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:32.516000 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet 011cfc9512c54fe689ca5ba948e57fd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:32.516081 10519 tablet_replica.cc:330] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:32.515751 9894 raft_consensus.cc:2229] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.516202 10519 raft_consensus.cc:2229] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:32.516268 10238 tablet_replica.cc:330] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:32.516291 9894 raft_consensus.cc:2258] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.516376 10238 raft_consensus.cc:2229] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.516470 10519 raft_consensus.cc:2258] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.516470 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbf45071dd240c2ab6e75a9beac5b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:32.516499 10238 raft_consensus.cc:2258] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.516650 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet bcbf45071dd240c2ab6e75a9beac5b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:32.516992 10238 ts_tablet_manager.cc:1875] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.516984 9529 tablet_service.cc:1514] Processing DeleteTablet for tablet d47b2dc234b342119e5dafea80caecbb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:32.517338 10519 ts_tablet_manager.cc:1875] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.517371 10687 tablet_replica.cc:330] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:32.517489 10687 raft_consensus.cc:2229] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.517577 10687 raft_consensus.cc:2258] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.517685 9894 ts_tablet_manager.cc:1875] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.518178 10687 ts_tablet_manager.cc:1875] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.519227 10233 ts_tablet_manager.cc:902] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:32.519212 10238 ts_tablet_manager.cc:1888] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.519284 10238 log.cc:1196] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/bcbf45071dd240c2ab6e75a9beac5b31
I20240426 09:49:32.519558 10238 ts_tablet_manager.cc:1909] T bcbf45071dd240c2ab6e75a9beac5b31 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:32.519675 10233 tablet_copy_client.cc:308] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:32.519832 9894 ts_tablet_manager.cc:1888] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.519892 9894 log.cc:1196] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/d47b2dc234b342119e5dafea80caecbb
I20240426 09:49:32.519970 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 011cfc9512c54fe689ca5ba948e57fd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:32 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:32.520112 10238 tablet_replica.cc:330] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:32.520198 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet bcbf45071dd240c2ab6e75a9beac5b31 (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.520244 10238 raft_consensus.cc:2229] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.520378 10238 raft_consensus.cc:2258] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.520510 9894 ts_tablet_manager.cc:1909] T d47b2dc234b342119e5dafea80caecbb P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:32.520766 10519 ts_tablet_manager.cc:1888] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.520823 10519 log.cc:1196] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/d47b2dc234b342119e5dafea80caecbb
I20240426 09:49:32.520843 10238 ts_tablet_manager.cc:1875] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.521068 9894 tablet_replica.cc:330] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:32.521087 10519 ts_tablet_manager.cc:1909] T d47b2dc234b342119e5dafea80caecbb P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:32.521142 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet d47b2dc234b342119e5dafea80caecbb (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.521205 9894 raft_consensus.cc:2229] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.521330 9894 raft_consensus.cc:2258] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.521616 10519 tablet_replica.cc:330] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:32.521700 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet d47b2dc234b342119e5dafea80caecbb (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.521732 10519 raft_consensus.cc:2229] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.521757 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet b2d42c52b7a94e32ab4cdad78ce884ee from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:40185)
I20240426 09:49:32.521833 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet b2d42c52b7a94e32ab4cdad78ce884ee from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:40185: session id = a692602e1a2e4794877e6fb92a7b9962-b2d42c52b7a94e32ab4cdad78ce884ee
I20240426 09:49:32.521836 10519 raft_consensus.cc:2258] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.521890 10687 ts_tablet_manager.cc:1888] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.521946 10687 log.cc:1196] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/bcbf45071dd240c2ab6e75a9beac5b31
I20240426 09:49:32.522280 10519 ts_tablet_manager.cc:1875] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.522459 10687 ts_tablet_manager.cc:1909] T bcbf45071dd240c2ab6e75a9beac5b31 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:32.523097 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet bcbf45071dd240c2ab6e75a9beac5b31 (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.523116 10238 ts_tablet_manager.cc:1888] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:32.523180 10238 log.cc:1196] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/011cfc9512c54fe689ca5ba948e57fd8
I20240426 09:49:32.523509 10238 ts_tablet_manager.cc:1909] T 011cfc9512c54fe689ca5ba948e57fd8 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:32.524098 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 011cfc9512c54fe689ca5ba948e57fd8 (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.524287 10519 ts_tablet_manager.cc:1888] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:32.524348 10519 log.cc:1196] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/011cfc9512c54fe689ca5ba948e57fd8
I20240426 09:49:32.524348 7295 tablet_copy_source_session.cc:205] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.524818 10519 ts_tablet_manager.cc:1909] T 011cfc9512c54fe689ca5ba948e57fd8 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:32.525156 10233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d42c52b7a94e32ab4cdad78ce884ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.525465 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 011cfc9512c54fe689ca5ba948e57fd8 (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.526805 9894 ts_tablet_manager.cc:1875] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.527060 10233 tablet_copy_client.cc:783] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.527215 10233 tablet_copy_client.cc:647] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.528224 10233 tablet_copy_client.cc:523] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.528998 9894 ts_tablet_manager.cc:1888] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.529078 9894 log.cc:1196] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/bcbf45071dd240c2ab6e75a9beac5b31
I20240426 09:49:32.529338 9894 ts_tablet_manager.cc:1909] T bcbf45071dd240c2ab6e75a9beac5b31 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:32.529429 10233 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:32.529906 9894 tablet_replica.cc:330] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:32.529963 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet bcbf45071dd240c2ab6e75a9beac5b31 (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.530032 9894 raft_consensus.cc:2229] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:32.530210 9894 raft_consensus.cc:2258] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.530689 9894 ts_tablet_manager.cc:1875] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.536270 9894 ts_tablet_manager.cc:1888] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:32.536341 9894 log.cc:1196] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/011cfc9512c54fe689ca5ba948e57fd8
I20240426 09:49:32.536567 9894 ts_tablet_manager.cc:1909] T 011cfc9512c54fe689ca5ba948e57fd8 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:32.537220 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 011cfc9512c54fe689ca5ba948e57fd8 (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.537524 10689 ts_tablet_manager.cc:1875] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.538946 10690 tablet_replica.cc:330] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:32.539132 10690 raft_consensus.cc:2229] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.539333 10690 raft_consensus.cc:2258] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.539721 10689 ts_tablet_manager.cc:1888] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.539832 10689 log.cc:1196] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/d47b2dc234b342119e5dafea80caecbb
I20240426 09:49:32.540125 10689 ts_tablet_manager.cc:1909] T d47b2dc234b342119e5dafea80caecbb P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:32.540267 10690 ts_tablet_manager.cc:1875] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.540834 10689 tablet_replica.cc:330] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:32.540933 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet d47b2dc234b342119e5dafea80caecbb (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.541016 10689 raft_consensus.cc:2229] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:32.541260 10689 raft_consensus.cc:2258] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.541827 10689 ts_tablet_manager.cc:1875] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:32.542970 10690 ts_tablet_manager.cc:1888] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.543099 10690 log.cc:1196] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/d47b2dc234b342119e5dafea80caecbb
I20240426 09:49:32.543419 10690 ts_tablet_manager.cc:1909] T d47b2dc234b342119e5dafea80caecbb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:32.543922 10689 ts_tablet_manager.cc:1888] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:32.544029 10689 log.cc:1196] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/bcbf45071dd240c2ab6e75a9beac5b31
I20240426 09:49:32.544232 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet d47b2dc234b342119e5dafea80caecbb (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.544365 10689 ts_tablet_manager.cc:1909] T bcbf45071dd240c2ab6e75a9beac5b31 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:32.545078 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet bcbf45071dd240c2ab6e75a9beac5b31 (table rebalance_test_table_8 [id=9d3d86954afa43f2b83e09268c25bff9]) successfully deleted
I20240426 09:49:32.547940 10345 ts_tablet_manager.cc:902] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:32.548377 10345 tablet_copy_client.cc:308] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:32.554041 9415 raft_consensus.cc:1203] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.554057 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 69179df98341493987f1d70ab24f62ef from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:55679)
I20240426 09:49:32.554152 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 69179df98341493987f1d70ab24f62ef from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:55679: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-69179df98341493987f1d70ab24f62ef
I20240426 09:49:32.555189 6902 tablet_copy_source_session.cc:205] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:32.555802 10233 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.556080 10233 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:32.556124 10345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69179df98341493987f1d70ab24f62ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.556257 10233 ts_tablet_manager.cc:1367] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.027s user 0.024s sys 0.001s
I20240426 09:49:32.556731 10233 raft_consensus.cc:348] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:32.556872 10233 raft_consensus.cc:729] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.557096 10233 consensus_queue.cc:260] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:32.557400 10233 ts_tablet_manager.cc:1398] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:32.558101 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-b2d42c52b7a94e32ab4cdad78ce884ee received from {username='slave'} at 127.0.99.133:40185
I20240426 09:49:32.558192 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-b2d42c52b7a94e32ab4cdad78ce884ee on tablet b2d42c52b7a94e32ab4cdad78ce884ee with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:32.558375 10345 tablet_copy_client.cc:783] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:32.558519 10345 tablet_copy_client.cc:647] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:32.560178 10345 tablet_copy_client.cc:523] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:32.561319 10345 tablet_bootstrap.cc:492] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:32.583536 10345 tablet_bootstrap.cc:492] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:32.583889 10345 tablet_bootstrap.cc:492] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:32.584072 10345 ts_tablet_manager.cc:1367] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.023s user 0.021s sys 0.000s
I20240426 09:49:32.584507 10164 raft_consensus.cc:482] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:32.584556 10345 raft_consensus.cc:348] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:32.584676 10345 raft_consensus.cc:729] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.584642 10164 raft_consensus.cc:504] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.584790 10345 consensus_queue.cc:260] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:32.585057 10345 ts_tablet_manager.cc:1398] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:32.585180 10164 leader_election.cc:288] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:32.585973 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-69179df98341493987f1d70ab24f62ef received from {username='slave'} at 127.0.99.135:55679
I20240426 09:49:32.586051 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-69179df98341493987f1d70ab24f62ef on tablet 69179df98341493987f1d70ab24f62ef with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:32.589987 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdcd6105a2647738257638e1e1096c3" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:32.590160 9415 raft_consensus.cc:2454] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:32.590230 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdcd6105a2647738257638e1e1096c3" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:32.590378 9286 raft_consensus.cc:2454] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:32.590492 9614 leader_election.cc:302] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:32.590835 10164 raft_consensus.cc:2781] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:32.590900 10164 raft_consensus.cc:482] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:32.590950 10164 raft_consensus.cc:3037] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.591993 10164 raft_consensus.cc:504] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.592360 10164 leader_election.cc:288] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:32.592679 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdcd6105a2647738257638e1e1096c3" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:32.592679 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdcd6105a2647738257638e1e1096c3" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:32.592811 9415 raft_consensus.cc:3037] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.592811 9286 raft_consensus.cc:3037] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.593780 9415 raft_consensus.cc:2454] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:32.593780 9286 raft_consensus.cc:2454] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:32.594072 9615 leader_election.cc:302] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:32.594246 10164 raft_consensus.cc:2781] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:32.594337 10164 raft_consensus.cc:686] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:32.594470 10164 consensus_queue.cc:237] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.596163 6732 catalog_manager.cc:5484] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:32.602998 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:32.607528 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff389ca828d4685b1e27f31123e6382 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=354c75c71828491a95ade6451ab52b89]), partition=
I20240426 09:49:32.607617 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff389ca828d4685b1e27f31123e6382 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=354c75c71828491a95ade6451ab52b89]), partition=
I20240426 09:49:32.607617 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff389ca828d4685b1e27f31123e6382 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=354c75c71828491a95ade6451ab52b89]), partition=
I20240426 09:49:32.607795 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff389ca828d4685b1e27f31123e6382. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.607851 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff389ca828d4685b1e27f31123e6382. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.607851 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff389ca828d4685b1e27f31123e6382. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:32.609426 10648 tablet_bootstrap.cc:492] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:32.609809 10160 tablet_bootstrap.cc:492] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:32.610049 10704 tablet_bootstrap.cc:492] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:32.610381 10648 tablet_bootstrap.cc:654] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.610677 10160 tablet_bootstrap.cc:654] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.610891 10704 tablet_bootstrap.cc:654] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:32.611680 10648 tablet_bootstrap.cc:492] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:32.611757 10648 ts_tablet_manager.cc:1367] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:32.612071 10704 tablet_bootstrap.cc:492] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:32.612135 10704 ts_tablet_manager.cc:1367] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:32.612113 10648 raft_consensus.cc:348] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.612205 10648 raft_consensus.cc:374] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.612227 10648 raft_consensus.cc:729] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.612331 10648 consensus_queue.cc:260] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.612470 10704 raft_consensus.cc:348] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.612566 10648 ts_tablet_manager.cc:1398] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:32.612567 10704 raft_consensus.cc:374] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.612598 10704 raft_consensus.cc:729] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.612712 10704 consensus_queue.cc:260] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.613082 10704 ts_tablet_manager.cc:1398] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:32.613752 10160 tablet_bootstrap.cc:492] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:32.613839 10160 ts_tablet_manager.cc:1367] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:32.614301 10160 raft_consensus.cc:348] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.614411 10160 raft_consensus.cc:374] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:32.614468 10160 raft_consensus.cc:729] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:32.614576 10160 consensus_queue.cc:260] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.614842 10160 ts_tablet_manager.cc:1398] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:32.627173 10164 raft_consensus.cc:482] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:32.627272 10164 raft_consensus.cc:504] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.627653 10164 leader_election.cc:288] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:32.627926 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff389ca828d4685b1e27f31123e6382" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:32.628064 9415 raft_consensus.cc:2454] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:32.628041 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff389ca828d4685b1e27f31123e6382" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:32.628182 7275 raft_consensus.cc:2454] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:32.628306 9614 leader_election.cc:302] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:32.628469 10164 raft_consensus.cc:2781] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:32.628530 10164 raft_consensus.cc:482] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:32.628576 10164 raft_consensus.cc:3037] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.629498 10164 raft_consensus.cc:504] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.629803 10164 leader_election.cc:288] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:32.630105 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff389ca828d4685b1e27f31123e6382" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:32.630172 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff389ca828d4685b1e27f31123e6382" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:32.630216 9415 raft_consensus.cc:3037] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.630277 7275 raft_consensus.cc:3037] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.631204 9415 raft_consensus.cc:2454] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:32.631204 7275 raft_consensus.cc:2454] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:32.631484 9614 leader_election.cc:302] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:32.631652 10164 raft_consensus.cc:2781] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:32.631718 10164 raft_consensus.cc:686] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:32.631848 10164 consensus_queue.cc:237] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.633522 6732 catalog_manager.cc:5484] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:32.646576 10164 raft_consensus.cc:482] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:32.646682 10164 raft_consensus.cc:504] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.647157 10164 leader_election.cc:288] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:32.647518 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce43c4c9b2994813ad7ad38f2e1b1040" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:32.647559 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce43c4c9b2994813ad7ad38f2e1b1040" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:32.647644 6880 raft_consensus.cc:2454] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:32.647720 9286 raft_consensus.cc:2454] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:32.647950 9616 leader_election.cc:302] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:32.648176 10164 raft_consensus.cc:2781] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:32.648223 10164 raft_consensus.cc:482] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:32.648248 10164 raft_consensus.cc:3037] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.649085 10164 raft_consensus.cc:504] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.649556 10164 leader_election.cc:288] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:32.649747 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce43c4c9b2994813ad7ad38f2e1b1040" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:32.649848 6880 raft_consensus.cc:3037] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.649930 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce43c4c9b2994813ad7ad38f2e1b1040" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:32.650035 9286 raft_consensus.cc:3037] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:32.650732 6880 raft_consensus.cc:2454] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:32.650856 9286 raft_consensus.cc:2454] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:32.651110 9615 leader_election.cc:302] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:32.651262 10164 raft_consensus.cc:2781] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:32.651341 10164 raft_consensus.cc:686] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:32.651479 10164 consensus_queue.cc:237] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:32.653100 6732 catalog_manager.cc:5484] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:32.657896 9672 raft_consensus.cc:1203] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.687979 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:32.688314 6991 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:32.701759 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:32.702627 6861 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:32.704051 9661 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:32.704993 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:32.720018 7122 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:32.723081 9529 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:32.743173 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:32.748019 9415 raft_consensus.cc:1261] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:32.748631 7275 raft_consensus.cc:1261] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:32.749022 10201 consensus_queue.cc:1035] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:32.750351 10164 consensus_queue.cc:1035] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:32.753528 9286 raft_consensus.cc:1203] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.754525 6737 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 9ff389ca828d4685b1e27f31123e6382 (table rebalancer_dynamic_table [id=354c75c71828491a95ade6451ab52b89]). Expected version 1 got 0
I20240426 09:49:32.756475 6737 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 9ff389ca828d4685b1e27f31123e6382 (table rebalancer_dynamic_table [id=354c75c71828491a95ade6451ab52b89]). Expected version 1 got 0
I20240426 09:49:32.758738 10231 tablet.cc:1722] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:32.759104 10231 tablet.cc:1620] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:32.762220 6732 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 9ff389ca828d4685b1e27f31123e6382 (table rebalancer_dynamic_table [id=354c75c71828491a95ade6451ab52b89]). Expected version 1 got 0
I20240426 09:49:32.763000 6733 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 9ff389ca828d4685b1e27f31123e6382 (table rebalancer_dynamic_table [id=354c75c71828491a95ade6451ab52b89]). Expected version 1 got 0
I20240426 09:49:32.765105 10728 mvcc.cc:205] Tried to move back new op lower bound from 7021055888371007488 to 7021055887901327360. Current Snapshot: MvccSnapshot[applied={T|T < 7021055888371007488}]
I20240426 09:49:32.765247 10731 tablet.cc:1722] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:32.766162 10730 tablet.cc:1722] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:32.767391 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=354c75c71828491a95ade6451ab52b89] alter complete (version 1)
I20240426 09:49:32.767997 10730 tablet.cc:1620] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:32.766808 10731 tablet.cc:1620] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:32.793226 9672 raft_consensus.cc:1203] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.819638 9548 raft_consensus.cc:1203] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.854537 6991 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:32.854935 7122 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:32.866415 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:32.870234 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:32.870434 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:32.871922 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:32.878437 10231 tablet.cc:1722] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:32.878731 10231 tablet.cc:1620] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:32.878948 10731 tablet.cc:1722] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:32.879240 10731 tablet.cc:1620] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:32.879976 9661 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:32.881960 10730 tablet.cc:1722] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:32.882341 10730 tablet.cc:1620] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:32.886993 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=354c75c71828491a95ade6451ab52b89] alter complete (version 2)
I20240426 09:49:32.901347 9415 raft_consensus.cc:1203] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.905520 9529 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:32.905542 6861 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:32.908095 9415 raft_consensus.cc:1203] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.927686 9672 raft_consensus.cc:1203] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.941071 9286 raft_consensus.cc:1203] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:32.961687 7275 consensus_queue.cc:237] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:32.962870 6880 raft_consensus.cc:1261] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:32.962949 9548 raft_consensus.cc:1261] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:32.963270 10008 consensus_queue.cc:1035] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:32.963567 10008 consensus_queue.cc:1035] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:32.965077 10008 raft_consensus.cc:2932] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:32.965536 6880 raft_consensus.cc:2932] T ce1e72dc91434c4ab63639fcc50da2ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:32.966667 9548 raft_consensus.cc:2932] T ce1e72dc91434c4ab63639fcc50da2ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:32.968236 6735 catalog_manager.cc:5484] T ce1e72dc91434c4ab63639fcc50da2ec P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:32.972399 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet ce1e72dc91434c4ab63639fcc50da2ec with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:32.972640 10689 tablet_replica.cc:330] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:32.972769 10689 raft_consensus.cc:2229] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:32.972913 10689 raft_consensus.cc:2258] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:32.973419 10689 ts_tablet_manager.cc:1875] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:32.975522 10689 ts_tablet_manager.cc:1888] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:32.975641 10689 log.cc:1196] T ce1e72dc91434c4ab63639fcc50da2ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/ce1e72dc91434c4ab63639fcc50da2ec
I20240426 09:49:32.976393 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet ce1e72dc91434c4ab63639fcc50da2ec (table rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]) successfully deleted
I20240426 09:49:32.994953 10196 consensus_queue.cc:1035] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:32.998173 10196 consensus_queue.cc:1035] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:32.998242 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:33.004207 10231 tablet.cc:1722] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:33.005595 10731 tablet.cc:1722] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:33.005635 10730 tablet.cc:1722] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:33.011165 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=354c75c71828491a95ade6451ab52b89] alter complete (version 3)
I20240426 09:49:33.014861 6737 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240426 09:49:33.014973 6737 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240426 09:49:33.017041 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 274ef2053af94bf1a038ec146c047063
I20240426 09:49:33.017362 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 3f47b3eff1504b4d838c2afc3c0bb287
I20240426 09:49:33.017695 6861 tablet_service.cc:1514] Processing DeleteTablet for tablet 274ef2053af94bf1a038ec146c047063 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:33.017773 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet 274ef2053af94bf1a038ec146c047063 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:33.017913 9894 tablet_replica.cc:330] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:33.017916 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 274ef2053af94bf1a038ec146c047063 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:33.017963 10238 tablet_replica.cc:330] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:33.018070 10689 tablet_replica.cc:330] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:33.018070 9894 raft_consensus.cc:2229] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.018085 10238 raft_consensus.cc:2229] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:33.018199 9894 raft_consensus.cc:2258] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.018193 10689 raft_consensus.cc:2229] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.018273 10238 raft_consensus.cc:2258] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.018322 10689 raft_consensus.cc:2258] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.018719 10238 ts_tablet_manager.cc:1875] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.018726 9894 ts_tablet_manager.cc:1875] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.018810 10689 ts_tablet_manager.cc:1875] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.020359 10238 ts_tablet_manager.cc:1888] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.020426 10238 log.cc:1196] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/274ef2053af94bf1a038ec146c047063
I20240426 09:49:33.020656 10238 ts_tablet_manager.cc:1909] T 274ef2053af94bf1a038ec146c047063 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:33.020833 10689 ts_tablet_manager.cc:1888] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.020833 9894 ts_tablet_manager.cc:1888] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.020898 10689 log.cc:1196] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/274ef2053af94bf1a038ec146c047063
I20240426 09:49:33.020906 9894 log.cc:1196] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/274ef2053af94bf1a038ec146c047063
I20240426 09:49:33.021163 9894 ts_tablet_manager.cc:1909] T 274ef2053af94bf1a038ec146c047063 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:33.021170 10689 ts_tablet_manager.cc:1909] T 274ef2053af94bf1a038ec146c047063 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:33.021248 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 274ef2053af94bf1a038ec146c047063 (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.021745 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f47b3eff1504b4d838c2afc3c0bb287 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:33.021829 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 274ef2053af94bf1a038ec146c047063 (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.022136 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 4 replicas of tablet 69179df98341493987f1d70ab24f62ef
I20240426 09:49:33.022333 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f47b3eff1504b4d838c2afc3c0bb287 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:33.022480 10238 tablet_replica.cc:330] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:33.022603 10238 raft_consensus.cc:2229] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.022637 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f47b3eff1504b4d838c2afc3c0bb287 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:33.022754 10238 raft_consensus.cc:2258] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.022879 10519 tablet_replica.cc:330] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:33.022924 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 274ef2053af94bf1a038ec146c047063 (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.023008 10519 raft_consensus.cc:2229] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:33.023233 10519 raft_consensus.cc:2258] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.023285 9894 tablet_replica.cc:330] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:33.023326 10238 ts_tablet_manager.cc:1875] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.023422 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 69179df98341493987f1d70ab24f62ef with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:33.023757 10519 ts_tablet_manager.cc:1875] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.023983 9894 raft_consensus.cc:2229] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.023996 9529 tablet_service.cc:1514] Processing DeleteTablet for tablet 69179df98341493987f1d70ab24f62ef with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:33.024111 9894 raft_consensus.cc:2258] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.024528 9894 ts_tablet_manager.cc:1875] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.024992 10238 ts_tablet_manager.cc:1888] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.025040 10238 log.cc:1196] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/3f47b3eff1504b4d838c2afc3c0bb287
I20240426 09:49:33.025254 10238 ts_tablet_manager.cc:1909] T 3f47b3eff1504b4d838c2afc3c0bb287 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:33.025597 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet 69179df98341493987f1d70ab24f62ef with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:33.025959 10519 ts_tablet_manager.cc:1888] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.026006 10238 tablet_replica.cc:330] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:33.026033 10519 log.cc:1196] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/3f47b3eff1504b4d838c2afc3c0bb287
I20240426 09:49:33.026120 10238 raft_consensus.cc:2229] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.026165 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 3f47b3eff1504b4d838c2afc3c0bb287 (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.026296 10519 ts_tablet_manager.cc:1909] T 3f47b3eff1504b4d838c2afc3c0bb287 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:33.026294 10238 raft_consensus.cc:2258] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.026638 10238 ts_tablet_manager.cc:1875] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.026921 10690 tablet_replica.cc:330] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:33.026976 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 3f47b3eff1504b4d838c2afc3c0bb287 (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.027043 9894 ts_tablet_manager.cc:1888] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.027092 10690 raft_consensus.cc:2229] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.027096 9894 log.cc:1196] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/3f47b3eff1504b4d838c2afc3c0bb287
I20240426 09:49:33.027206 10690 raft_consensus.cc:2258] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.027480 9894 ts_tablet_manager.cc:1909] T 3f47b3eff1504b4d838c2afc3c0bb287 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:33.027829 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 69179df98341493987f1d70ab24f62ef with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:33.027978 10689 tablet_replica.cc:330] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:33.027977 10690 ts_tablet_manager.cc:1875] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.028024 9894 tablet_replica.cc:330] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:33.028091 10689 raft_consensus.cc:2229] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.028123 9894 raft_consensus.cc:2229] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:33.028249 10689 raft_consensus.cc:2258] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.028441 9894 raft_consensus.cc:2258] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.028515 10238 ts_tablet_manager.cc:1888] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.028573 10238 log.cc:1196] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/69179df98341493987f1d70ab24f62ef
I20240426 09:49:33.028762 10689 ts_tablet_manager.cc:1875] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.028829 10238 ts_tablet_manager.cc:1909] T 69179df98341493987f1d70ab24f62ef P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:33.028882 9894 ts_tablet_manager.cc:1875] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.030022 10690 ts_tablet_manager.cc:1888] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.030074 10690 log.cc:1196] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/69179df98341493987f1d70ab24f62ef
I20240426 09:49:33.030261 10690 ts_tablet_manager.cc:1909] T 69179df98341493987f1d70ab24f62ef P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:33.030546 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 69179df98341493987f1d70ab24f62ef (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.030663 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 3f47b3eff1504b4d838c2afc3c0bb287 (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.030707 9894 ts_tablet_manager.cc:1888] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.030707 10689 ts_tablet_manager.cc:1888] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.030789 9894 log.cc:1196] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/69179df98341493987f1d70ab24f62ef
I20240426 09:49:33.030815 10689 log.cc:1196] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/69179df98341493987f1d70ab24f62ef
I20240426 09:49:33.031059 10689 ts_tablet_manager.cc:1909] T 69179df98341493987f1d70ab24f62ef P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:33.031059 9894 ts_tablet_manager.cc:1909] T 69179df98341493987f1d70ab24f62ef P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:33.031163 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 69179df98341493987f1d70ab24f62ef (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.031874 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 69179df98341493987f1d70ab24f62ef (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.033084 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 69179df98341493987f1d70ab24f62ef (table rebalance_test_table_10 [id=1d62dba75f7c4959b029b44dd4e4bdbd]) successfully deleted
I20240426 09:49:33.068351 7122 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:33.068343 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:33.076828 6991 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:33.078945 9661 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:33.079023 6862 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:33.087456 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:33.091503 7253 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:33.099496 9529 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:33.113919 6738 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:33.114068 6738 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:33.116201 6738 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 9ff389ca828d4685b1e27f31123e6382
I20240426 09:49:33.116811 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff389ca828d4685b1e27f31123e6382 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:33.117107 10857 tablet_replica.cc:330] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:33.117255 10857 raft_consensus.cc:2229] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.117338 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff389ca828d4685b1e27f31123e6382 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:33.117390 10857 raft_consensus.cc:2258] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.117794 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff389ca828d4685b1e27f31123e6382 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:33.117969 10238 tablet_replica.cc:330] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:33.118005 10857 ts_tablet_manager.cc:1875] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.118119 10238 raft_consensus.cc:2229] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.118263 10238 raft_consensus.cc:2258] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.118932 10238 ts_tablet_manager.cc:1875] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.120393 10857 ts_tablet_manager.cc:1888] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.120471 10857 log.cc:1196] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/9ff389ca828d4685b1e27f31123e6382
I20240426 09:49:33.120728 10857 ts_tablet_manager.cc:1909] T 9ff389ca828d4685b1e27f31123e6382 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:33.121155 10238 ts_tablet_manager.cc:1888] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.121229 10238 log.cc:1196] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/9ff389ca828d4685b1e27f31123e6382
I20240426 09:49:33.121428 10858 tablet_replica.cc:330] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:33.121583 10858 raft_consensus.cc:2229] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:33.121634 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 9ff389ca828d4685b1e27f31123e6382 (table rebalancer_dynamic_table_new_name [id=354c75c71828491a95ade6451ab52b89]) successfully deleted
I20240426 09:49:33.121954 10858 raft_consensus.cc:2258] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.122617 10238 ts_tablet_manager.cc:1909] T 9ff389ca828d4685b1e27f31123e6382 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:33.122833 10164 consensus_queue.cc:1035] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.123387 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 9ff389ca828d4685b1e27f31123e6382 (table rebalancer_dynamic_table_new_name [id=354c75c71828491a95ade6451ab52b89]) successfully deleted
I20240426 09:49:33.124193 10858 ts_tablet_manager.cc:1875] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.127207 10201 consensus_queue.cc:1035] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.131847 10858 ts_tablet_manager.cc:1888] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.131978 10858 log.cc:1196] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/9ff389ca828d4685b1e27f31123e6382
I20240426 09:49:33.132318 10858 ts_tablet_manager.cc:1909] T 9ff389ca828d4685b1e27f31123e6382 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:33.133859 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 9ff389ca828d4685b1e27f31123e6382 (table rebalancer_dynamic_table_new_name [id=354c75c71828491a95ade6451ab52b89]) successfully deleted
I20240426 09:49:33.157460 7275 consensus_queue.cc:237] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.158653 9548 raft_consensus.cc:1261] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:33.158816 7143 raft_consensus.cc:1261] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:33.159097 10201 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.159232 10802 consensus_queue.cc:1035] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:33.159487 10802 consensus_queue.cc:1035] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:33.160732 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:33.162432 10365 raft_consensus.cc:2932] T 9a9c1e00452f4f32b31e8b336892df49 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.164513 9548 raft_consensus.cc:2932] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.166241 10201 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.168591 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 10118b6d060f4999a2cdfdaa1b8de197 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.168888 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10118b6d060f4999a2cdfdaa1b8de197. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.168998 9529 tablet_service.cc:1467] Processing CreateTablet for tablet e3114093f8f24864b7184f0bd91fb401 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.168995 6993 tablet_service.cc:1467] Processing CreateTablet for tablet e3114093f8f24864b7184f0bd91fb401 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.169239 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3114093f8f24864b7184f0bd91fb401. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.169268 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3114093f8f24864b7184f0bd91fb401. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.169977 6737 catalog_manager.cc:5484] T 9a9c1e00452f4f32b31e8b336892df49 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.171034 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 10118b6d060f4999a2cdfdaa1b8de197 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.171288 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10118b6d060f4999a2cdfdaa1b8de197. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.172425 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 10118b6d060f4999a2cdfdaa1b8de197 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.172672 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10118b6d060f4999a2cdfdaa1b8de197. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.173779 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 8e306113f9874277a60468d9248d45ce (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.173986 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e306113f9874277a60468d9248d45ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.176774 9397 tablet_service.cc:1467] Processing CreateTablet for tablet e3114093f8f24864b7184f0bd91fb401 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.177021 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3114093f8f24864b7184f0bd91fb401. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.177183 9395 tablet_service.cc:1467] Processing CreateTablet for tablet 8e306113f9874277a60468d9248d45ce (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.177402 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e306113f9874277a60468d9248d45ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.179468 10885 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:33.180333 10885 tablet_bootstrap.cc:654] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.182127 7142 raft_consensus.cc:2932] T 9a9c1e00452f4f32b31e8b336892df49 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.183455 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 8e306113f9874277a60468d9248d45ce (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.183758 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e306113f9874277a60468d9248d45ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.186475 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a9c1e00452f4f32b31e8b336892df49 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:33.186851 10689 tablet_replica.cc:330] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:33.187000 10689 raft_consensus.cc:2229] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.187140 10689 raft_consensus.cc:2258] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.187636 10689 ts_tablet_manager.cc:1875] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:33.188473 10884 tablet_bootstrap.cc:492] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:33.188747 10883 tablet_bootstrap.cc:492] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:33.189065 10882 tablet_bootstrap.cc:492] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:33.189558 10689 ts_tablet_manager.cc:1888] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:33.189612 10689 log.cc:1196] T 9a9c1e00452f4f32b31e8b336892df49 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/9a9c1e00452f4f32b31e8b336892df49
I20240426 09:49:33.190135 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 9a9c1e00452f4f32b31e8b336892df49 (table rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]) successfully deleted
I20240426 09:49:33.190194 10883 tablet_bootstrap.cc:654] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.191229 10882 tablet_bootstrap.cc:654] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.191447 10884 tablet_bootstrap.cc:654] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.191529 10885 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:33.191625 10885 ts_tablet_manager.cc:1367] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240426 09:49:33.192178 10885 raft_consensus.cc:348] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.192302 10885 raft_consensus.cc:374] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.192341 10885 raft_consensus.cc:729] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.192494 10885 consensus_queue.cc:260] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.192775 10885 ts_tablet_manager.cc:1398] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.192994 10885 tablet_bootstrap.cc:492] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:33.193883 10885 tablet_bootstrap.cc:654] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.194598 10882 tablet_bootstrap.cc:492] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:33.195410 10883 tablet_bootstrap.cc:492] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:33.195500 10883 ts_tablet_manager.cc:1367] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:33.194927 10882 ts_tablet_manager.cc:1367] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:33.195920 10883 raft_consensus.cc:348] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:33.196041 10883 raft_consensus.cc:374] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.196079 10883 raft_consensus.cc:729] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.196221 10883 consensus_queue.cc:260] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:33.197225 10884 tablet_bootstrap.cc:492] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:33.197327 10885 tablet_bootstrap.cc:492] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:33.197353 10884 ts_tablet_manager.cc:1367] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240426 09:49:33.197480 10885 ts_tablet_manager.cc:1367] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:33.197623 10882 raft_consensus.cc:348] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.197958 10885 raft_consensus.cc:348] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.198105 10885 raft_consensus.cc:374] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.198168 10885 raft_consensus.cc:729] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.198302 10885 consensus_queue.cc:260] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.198608 10885 ts_tablet_manager.cc:1398] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.198664 10882 raft_consensus.cc:374] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.198868 10885 tablet_bootstrap.cc:492] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:33.199025 10883 ts_tablet_manager.cc:1398] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:33.199018 10882 raft_consensus.cc:729] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.199407 10882 consensus_queue.cc:260] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.199294 10884 raft_consensus.cc:348] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:33.199748 10883 tablet_bootstrap.cc:492] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:33.200299 10885 tablet_bootstrap.cc:654] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.200619 10883 tablet_bootstrap.cc:654] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.201462 10884 raft_consensus.cc:374] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.201712 10884 raft_consensus.cc:729] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.201902 10883 tablet_bootstrap.cc:492] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:33.201902 10882 ts_tablet_manager.cc:1398] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:33.202032 10883 ts_tablet_manager.cc:1367] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:33.202114 10884 consensus_queue.cc:260] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:33.202164 10885 tablet_bootstrap.cc:492] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:33.202242 10885 ts_tablet_manager.cc:1367] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:33.202657 10883 raft_consensus.cc:348] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.202668 10885 raft_consensus.cc:348] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:33.202793 10885 raft_consensus.cc:374] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.202811 10883 raft_consensus.cc:374] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.202836 10885 raft_consensus.cc:729] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.202857 10883 raft_consensus.cc:729] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.203003 10882 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:33.202956 10885 consensus_queue.cc:260] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:33.202972 10883 consensus_queue.cc:260] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.203228 10885 ts_tablet_manager.cc:1398] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.203841 10883 ts_tablet_manager.cc:1398] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:33.204949 10884 ts_tablet_manager.cc:1398] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:49:33.205823 10884 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:33.206957 10882 tablet_bootstrap.cc:654] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.207548 10884 tablet_bootstrap.cc:654] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.208839 10884 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:33.208938 10884 ts_tablet_manager.cc:1367] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:33.209015 10882 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:33.209092 10882 ts_tablet_manager.cc:1367] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240426 09:49:33.209342 10884 raft_consensus.cc:348] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.209453 10884 raft_consensus.cc:374] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.209501 10884 raft_consensus.cc:729] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.209496 10882 raft_consensus.cc:348] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.209596 10882 raft_consensus.cc:374] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.209669 10882 raft_consensus.cc:729] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.209630 10884 consensus_queue.cc:260] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.209800 10882 consensus_queue.cc:260] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.209990 10884 ts_tablet_manager.cc:1398] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.210819 10882 ts_tablet_manager.cc:1398] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:33.217589 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:33.217904 9917 raft_consensus.cc:482] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:33.217985 9917 raft_consensus.cc:504] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.218458 9917 leader_election.cc:288] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:33.224123 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3114093f8f24864b7184f0bd91fb401" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:33.224279 7011 raft_consensus.cc:2454] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:33.224562 9396 tablet_service.cc:1467] Processing CreateTablet for tablet fddda654ba204ac4b59e1cfc3a10ce20 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=597baef885c34141ae1f6a12a3a42832]), partition=
I20240426 09:49:33.224680 9482 leader_election.cc:302] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:33.224807 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fddda654ba204ac4b59e1cfc3a10ce20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.224838 9917 raft_consensus.cc:2781] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:33.224885 9917 raft_consensus.cc:482] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:33.224934 9917 raft_consensus.cc:3037] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.225908 9917 raft_consensus.cc:504] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.226186 10201 raft_consensus.cc:482] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:33.226223 9917 leader_election.cc:288] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:33.226264 10201 raft_consensus.cc:504] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.226583 10201 leader_election.cc:288] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:33.226855 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3114093f8f24864b7184f0bd91fb401" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:33.226979 9415 raft_consensus.cc:2454] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:33.227031 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e306113f9874277a60468d9248d45ce" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:33.227133 9548 raft_consensus.cc:2454] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:33.227108 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3114093f8f24864b7184f0bd91fb401" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:33.227236 7011 raft_consensus.cc:3037] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.227351 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e306113f9874277a60468d9248d45ce" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:33.227488 9417 raft_consensus.cc:2454] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:33.227497 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3114093f8f24864b7184f0bd91fb401" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:33.227588 9416 raft_consensus.cc:3037] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.227811 9614 leader_election.cc:302] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:33.228042 10201 raft_consensus.cc:2781] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:33.228093 10201 raft_consensus.cc:482] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:33.228128 10201 raft_consensus.cc:3037] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.228255 7011 raft_consensus.cc:2454] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:33.228559 9416 raft_consensus.cc:2454] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:33.228755 9661 tablet_service.cc:1467] Processing CreateTablet for tablet fddda654ba204ac4b59e1cfc3a10ce20 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=597baef885c34141ae1f6a12a3a42832]), partition=
I20240426 09:49:33.229027 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fddda654ba204ac4b59e1cfc3a10ce20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.229048 10201 raft_consensus.cc:504] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.229324 10201 leader_election.cc:288] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:33.229669 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e306113f9874277a60468d9248d45ce" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:33.229782 9548 raft_consensus.cc:3037] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.230577 9548 raft_consensus.cc:2454] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:33.230906 10885 tablet_bootstrap.cc:492] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:33.231148 10884 tablet_bootstrap.cc:492] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:33.231912 10885 tablet_bootstrap.cc:654] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.231936 10884 tablet_bootstrap.cc:654] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.233131 9614 leader_election.cc:302] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:33.233338 10201 raft_consensus.cc:2781] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:33.233417 10201 raft_consensus.cc:686] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:33.233539 10201 consensus_queue.cc:237] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.233594 9266 tablet_service.cc:1467] Processing CreateTablet for tablet fddda654ba204ac4b59e1cfc3a10ce20 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=597baef885c34141ae1f6a12a3a42832]), partition=
I20240426 09:49:33.233855 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fddda654ba204ac4b59e1cfc3a10ce20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.233879 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e306113f9874277a60468d9248d45ce" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:33.233976 9416 raft_consensus.cc:3037] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.234143 10884 tablet_bootstrap.cc:492] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:33.234218 10884 ts_tablet_manager.cc:1367] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240426 09:49:33.234618 10884 raft_consensus.cc:348] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.234710 10884 raft_consensus.cc:374] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.234747 10884 raft_consensus.cc:729] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.234848 9416 raft_consensus.cc:2454] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:33.234887 10884 consensus_queue.cc:260] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.234947 9482 leader_election.cc:302] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:33.235226 9917 raft_consensus.cc:2781] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:33.235366 9917 raft_consensus.cc:686] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:33.235491 9917 consensus_queue.cc:237] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.236263 10884 ts_tablet_manager.cc:1398] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:33.236215 6738 catalog_manager.cc:5484] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.237052 10885 tablet_bootstrap.cc:492] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:33.237134 10885 ts_tablet_manager.cc:1367] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240426 09:49:33.237655 10885 raft_consensus.cc:348] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.237807 10885 raft_consensus.cc:374] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.237910 10885 raft_consensus.cc:729] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.237949 6740 catalog_manager.cc:5484] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.238106 10885 consensus_queue.cc:260] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.238371 10885 ts_tablet_manager.cc:1398] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.238588 10913 tablet_bootstrap.cc:492] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:33.240005 10913 tablet_bootstrap.cc:654] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.241235 10913 tablet_bootstrap.cc:492] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:33.241380 10913 ts_tablet_manager.cc:1367] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:33.241740 10913 raft_consensus.cc:348] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.241887 10913 raft_consensus.cc:374] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.241950 10913 raft_consensus.cc:729] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.242147 10913 consensus_queue.cc:260] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.243542 10913 ts_tablet_manager.cc:1398] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:33.245342 10622 raft_consensus.cc:482] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:33.245412 10622 raft_consensus.cc:504] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.245762 10622 leader_election.cc:288] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:33.246047 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddda654ba204ac4b59e1cfc3a10ce20" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:33.246088 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddda654ba204ac4b59e1cfc3a10ce20" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:33.246184 9416 raft_consensus.cc:2454] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:33.246215 9672 raft_consensus.cc:2454] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:33.246465 9219 leader_election.cc:302] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:33.246985 10622 raft_consensus.cc:2781] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:33.247041 10622 raft_consensus.cc:482] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:33.247076 10622 raft_consensus.cc:3037] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.248064 10622 raft_consensus.cc:504] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.248342 10622 leader_election.cc:288] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:33.248818 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddda654ba204ac4b59e1cfc3a10ce20" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:33.248947 9416 raft_consensus.cc:3037] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.248937 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fddda654ba204ac4b59e1cfc3a10ce20" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:33.249044 9672 raft_consensus.cc:3037] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.249806 9672 raft_consensus.cc:2454] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:33.249917 9416 raft_consensus.cc:2454] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:33.250145 9219 leader_election.cc:302] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:33.250319 10622 raft_consensus.cc:2781] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:33.250411 10622 raft_consensus.cc:686] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:33.250537 10622 consensus_queue.cc:237] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.252139 6738 catalog_manager.cc:5484] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:33.267026 10033 raft_consensus.cc:482] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:33.267138 10033 raft_consensus.cc:504] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:33.267578 10033 leader_election.cc:288] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:33.267990 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10118b6d060f4999a2cdfdaa1b8de197" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:33.268146 9416 raft_consensus.cc:2454] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:33.268100 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10118b6d060f4999a2cdfdaa1b8de197" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:33.268265 9672 raft_consensus.cc:2454] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:33.268500 6946 leader_election.cc:302] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:33.268766 10033 raft_consensus.cc:2781] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:33.268842 10033 raft_consensus.cc:482] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:33.268893 10033 raft_consensus.cc:3037] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.269788 10033 raft_consensus.cc:504] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:33.270152 10033 leader_election.cc:288] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:33.270502 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10118b6d060f4999a2cdfdaa1b8de197" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:33.270502 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10118b6d060f4999a2cdfdaa1b8de197" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:33.270604 9672 raft_consensus.cc:3037] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.270604 9416 raft_consensus.cc:3037] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.271548 9416 raft_consensus.cc:2454] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:33.271548 9672 raft_consensus.cc:2454] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:33.272279 6946 leader_election.cc:302] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:33.272547 10033 raft_consensus.cc:2781] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:33.272614 10033 raft_consensus.cc:686] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:33.272733 10033 consensus_queue.cc:237] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:33.274915 6735 catalog_manager.cc:5484] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:33.304844 6993 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:33.305557 7122 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:33.306525 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:33.309993 9266 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:33.317296 6862 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:33.317808 9661 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:33.318809 9529 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:33.331146 7253 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:33.355247 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:33.359606 9672 raft_consensus.cc:1261] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:33.359687 9416 raft_consensus.cc:1261] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:33.360060 10622 consensus_queue.cc:1035] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.360350 10196 consensus_queue.cc:1035] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.362689 6738 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet fddda654ba204ac4b59e1cfc3a10ce20 (table rebalancer_dynamic_table [id=597baef885c34141ae1f6a12a3a42832]). Expected version 1 got 0
I20240426 09:49:33.363050 6733 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet fddda654ba204ac4b59e1cfc3a10ce20 (table rebalancer_dynamic_table [id=597baef885c34141ae1f6a12a3a42832]). Expected version 1 got 0
I20240426 09:49:33.372778 10231 tablet.cc:1722] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:33.373109 10231 tablet.cc:1620] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:33.376384 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=597baef885c34141ae1f6a12a3a42832] alter complete (version 1)
I20240426 09:49:33.380026 10731 tablet.cc:1722] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:33.380359 10731 tablet.cc:1620] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:33.380421 10960 tablet.cc:1722] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:33.380728 10960 tablet.cc:1620] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:33.403456 6880 consensus_queue.cc:237] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:33.404835 9286 raft_consensus.cc:1261] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:33.405226 7011 raft_consensus.cc:1261] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:33.405263 10725 consensus_queue.cc:1035] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:33.406087 10725 consensus_queue.cc:1035] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:33.407065 10282 raft_consensus.cc:2932] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:33.407537 9286 raft_consensus.cc:2932] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:33.408258 7011 raft_consensus.cc:2932] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:33.410990 6733 catalog_manager.cc:5484] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:33.413421 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dc287ff7ea640a38537c74e9ff156f7 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:33.413656 10519 tablet_replica.cc:330] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:33.413815 10519 raft_consensus.cc:2229] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.413947 10519 raft_consensus.cc:2258] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.414394 10519 ts_tablet_manager.cc:1875] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:33.417104 10519 ts_tablet_manager.cc:1888] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:33.417174 10519 log.cc:1196] T 6dc287ff7ea640a38537c74e9ff156f7 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/6dc287ff7ea640a38537c74e9ff156f7
I20240426 09:49:33.417632 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 6dc287ff7ea640a38537c74e9ff156f7 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:33.490574 7122 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:33.491050 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:33.491657 6993 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:33.494973 9266 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:33.498749 6862 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:33.508636 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:33.511811 7253 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:33.518900 10960 tablet.cc:1722] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:33.519227 10960 tablet.cc:1620] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:33.521829 9529 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:33.523281 10231 tablet.cc:1722] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:33.523593 10231 tablet.cc:1620] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:33.525430 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=597baef885c34141ae1f6a12a3a42832] alter complete (version 2)
I20240426 09:49:33.528733 10731 tablet.cc:1722] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:33.529014 10731 tablet.cc:1620] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:33.531353 6731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240426 09:49:33.531495 6731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240426 09:49:33.534605 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet c3dc11eb66744cb496ffbc2ff80ad571
I20240426 09:49:33.534940 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 19229a19d57c489fa224f5ce6a439485
I20240426 09:49:33.535068 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet c3dc11eb66744cb496ffbc2ff80ad571 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:33.535174 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 4 replicas of tablet f3b2b3e1d02b49269d19d284cab418da
I20240426 09:49:33.535315 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet c3dc11eb66744cb496ffbc2ff80ad571 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:33.535485 10238 tablet_replica.cc:330] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:33.535522 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 19229a19d57c489fa224f5ce6a439485 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:33.535636 10238 raft_consensus.cc:2229] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.535776 10238 raft_consensus.cc:2258] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.535946 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet c3dc11eb66744cb496ffbc2ff80ad571 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:33.536038 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 19229a19d57c489fa224f5ce6a439485 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:33.536109 10689 tablet_replica.cc:330] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:33.536218 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 19229a19d57c489fa224f5ce6a439485 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:33.536247 10689 raft_consensus.cc:2229] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.536301 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b2b3e1d02b49269d19d284cab418da with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:33.536339 10238 ts_tablet_manager.cc:1875] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.536382 10519 tablet_replica.cc:330] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:33.536382 10689 raft_consensus.cc:2258] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.536499 10519 raft_consensus.cc:2229] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.536633 10519 raft_consensus.cc:2258] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.536888 10689 ts_tablet_manager.cc:1875] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.537099 10519 ts_tablet_manager.cc:1875] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.538475 10238 ts_tablet_manager.cc:1888] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.538738 10238 log.cc:1196] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/c3dc11eb66744cb496ffbc2ff80ad571
I20240426 09:49:33.538901 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b2b3e1d02b49269d19d284cab418da with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:33.539072 10238 ts_tablet_manager.cc:1909] T c3dc11eb66744cb496ffbc2ff80ad571 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:33.539079 10858 tablet_replica.cc:330] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:33.539223 10858 raft_consensus.cc:2229] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.539256 10519 ts_tablet_manager.cc:1888] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.539311 10519 log.cc:1196] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/19229a19d57c489fa224f5ce6a439485
I20240426 09:49:33.539330 10858 raft_consensus.cc:2258] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.539574 10519 ts_tablet_manager.cc:1909] T 19229a19d57c489fa224f5ce6a439485 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:33.539937 10858 ts_tablet_manager.cc:1875] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.540153 10519 tablet_replica.cc:330] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:33.540230 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 19229a19d57c489fa224f5ce6a439485 (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.540277 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b2b3e1d02b49269d19d284cab418da with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:33.540483 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet c3dc11eb66744cb496ffbc2ff80ad571 (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.540655 10238 tablet_replica.cc:330] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:33.540711 10519 raft_consensus.cc:2229] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.540836 10238 raft_consensus.cc:2229] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:33.540892 10519 raft_consensus.cc:2258] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.541325 10238 raft_consensus.cc:2258] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.541457 10519 ts_tablet_manager.cc:1875] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.542016 10238 ts_tablet_manager.cc:1875] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.542052 10858 ts_tablet_manager.cc:1888] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.542135 10858 log.cc:1196] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/f3b2b3e1d02b49269d19d284cab418da
I20240426 09:49:33.542400 10858 ts_tablet_manager.cc:1909] T f3b2b3e1d02b49269d19d284cab418da P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:33.543394 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet f3b2b3e1d02b49269d19d284cab418da (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.543718 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet f3b2b3e1d02b49269d19d284cab418da with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:33.543962 10238 ts_tablet_manager.cc:1888] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.544026 10238 log.cc:1196] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/f3b2b3e1d02b49269d19d284cab418da
I20240426 09:49:33.544544 10689 ts_tablet_manager.cc:1888] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.544613 10689 log.cc:1196] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/c3dc11eb66744cb496ffbc2ff80ad571
I20240426 09:49:33.544863 10689 ts_tablet_manager.cc:1909] T c3dc11eb66744cb496ffbc2ff80ad571 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:33.545307 10238 ts_tablet_manager.cc:1909] T f3b2b3e1d02b49269d19d284cab418da P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:33.545493 10689 tablet_replica.cc:330] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:33.545594 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet c3dc11eb66744cb496ffbc2ff80ad571 (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.545621 10689 raft_consensus.cc:2229] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:33.545771 10689 raft_consensus.cc:2258] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.545966 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet f3b2b3e1d02b49269d19d284cab418da (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.546195 10689 ts_tablet_manager.cc:1875] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.547631 9661 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:33.548909 10519 ts_tablet_manager.cc:1888] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.548985 10519 log.cc:1196] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/f3b2b3e1d02b49269d19d284cab418da
I20240426 09:49:33.566033 10519 ts_tablet_manager.cc:1909] T f3b2b3e1d02b49269d19d284cab418da P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:33.566872 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet f3b2b3e1d02b49269d19d284cab418da (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.571084 10689 ts_tablet_manager.cc:1888] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.571169 10689 log.cc:1196] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/19229a19d57c489fa224f5ce6a439485
I20240426 09:49:33.571527 10689 ts_tablet_manager.cc:1909] T 19229a19d57c489fa224f5ce6a439485 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:33.572284 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 19229a19d57c489fa224f5ce6a439485 (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.574995 11030 tablet_replica.cc:330] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:33.575204 11030 raft_consensus.cc:2229] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:33.575459 11030 raft_consensus.cc:2258] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.577248 11030 ts_tablet_manager.cc:1875] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.580845 11030 ts_tablet_manager.cc:1888] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.580983 11030 log.cc:1196] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/c3dc11eb66744cb496ffbc2ff80ad571
I20240426 09:49:33.581307 11030 ts_tablet_manager.cc:1909] T c3dc11eb66744cb496ffbc2ff80ad571 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:33.581924 11030 tablet_replica.cc:330] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:33.582024 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet c3dc11eb66744cb496ffbc2ff80ad571 (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.582064 11030 raft_consensus.cc:2229] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.582188 11030 raft_consensus.cc:2258] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.582710 11030 ts_tablet_manager.cc:1875] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.584431 11030 ts_tablet_manager.cc:1888] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:33.584497 11030 log.cc:1196] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/19229a19d57c489fa224f5ce6a439485
I20240426 09:49:33.584720 11030 ts_tablet_manager.cc:1909] T 19229a19d57c489fa224f5ce6a439485 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:33.585188 11030 tablet_replica.cc:330] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:33.585309 11030 raft_consensus.cc:2229] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.585299 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 19229a19d57c489fa224f5ce6a439485 (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.585440 11030 raft_consensus.cc:2258] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.585826 11030 ts_tablet_manager.cc:1875] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.587440 11030 ts_tablet_manager.cc:1888] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.587507 11030 log.cc:1196] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/f3b2b3e1d02b49269d19d284cab418da
I20240426 09:49:33.587731 11030 ts_tablet_manager.cc:1909] T f3b2b3e1d02b49269d19d284cab418da P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:33.588251 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet f3b2b3e1d02b49269d19d284cab418da (table rebalance_test_table_12 [id=308333c7ec2e4ff8a81b7dfcb47b1ca8]) successfully deleted
I20240426 09:49:33.607825 7142 consensus_queue.cc:237] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.609028 6880 raft_consensus.cc:1261] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:33.609068 9548 raft_consensus.cc:1261] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:33.609421 9907 consensus_queue.cc:1035] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:33.609582 10894 consensus_queue.cc:1035] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:33.611310 10894 raft_consensus.cc:2932] T aca60cffb8cf41e3b0d9b64436c306d7 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.611783 6880 raft_consensus.cc:2932] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.613615 9548 raft_consensus.cc:2932] T aca60cffb8cf41e3b0d9b64436c306d7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.614806 6731 catalog_manager.cc:5484] T aca60cffb8cf41e3b0d9b64436c306d7 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.617290 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet aca60cffb8cf41e3b0d9b64436c306d7 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:33.617511 10689 tablet_replica.cc:330] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:33.617666 10689 raft_consensus.cc:2229] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.617853 10689 raft_consensus.cc:2258] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.618400 10689 ts_tablet_manager.cc:1875] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:33.620080 10689 ts_tablet_manager.cc:1888] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:33.620142 10689 log.cc:1196] T aca60cffb8cf41e3b0d9b64436c306d7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/aca60cffb8cf41e3b0d9b64436c306d7
I20240426 09:49:33.620680 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet aca60cffb8cf41e3b0d9b64436c306d7 (table rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]) successfully deleted
I20240426 09:49:33.637352 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:33.642702 10960 tablet.cc:1722] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:33.643324 10731 tablet.cc:1722] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:33.643576 10231 tablet.cc:1722] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:33.647820 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=597baef885c34141ae1f6a12a3a42832] alter complete (version 3)
I20240426 09:49:33.656546 10912 consensus_queue.cc:1035] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.660867 9917 consensus_queue.cc:1035] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.715049 6991 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:33.717586 7123 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:33.720539 9266 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:33.722630 6862 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:33.723784 10164 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.725095 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:33.726109 9529 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:33.727927 9661 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:33.729458 10201 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.740418 7253 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:33.749490 6739 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:33.749645 6739 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:33.751874 6739 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet fddda654ba204ac4b59e1cfc3a10ce20
I20240426 09:49:33.752454 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet fddda654ba204ac4b59e1cfc3a10ce20 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:33.752874 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet fddda654ba204ac4b59e1cfc3a10ce20 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:33.753032 10858 tablet_replica.cc:330] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:33.753150 10858 raft_consensus.cc:2229] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.753276 10858 raft_consensus.cc:2258] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.753975 10858 ts_tablet_manager.cc:1875] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.756127 10858 ts_tablet_manager.cc:1888] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.756188 10858 log.cc:1196] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/fddda654ba204ac4b59e1cfc3a10ce20
I20240426 09:49:33.756444 10858 ts_tablet_manager.cc:1909] T fddda654ba204ac4b59e1cfc3a10ce20 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:33.759461 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet fddda654ba204ac4b59e1cfc3a10ce20 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:33 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:33.765048 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet fddda654ba204ac4b59e1cfc3a10ce20 (table rebalancer_dynamic_table_new_name [id=597baef885c34141ae1f6a12a3a42832]) successfully deleted
I20240426 09:49:33.767537 11106 tablet_replica.cc:330] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:33.767692 11106 raft_consensus.cc:2229] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.767824 11106 raft_consensus.cc:2258] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.768370 11106 ts_tablet_manager.cc:1875] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.770045 11106 ts_tablet_manager.cc:1888] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.770119 11106 log.cc:1196] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/fddda654ba204ac4b59e1cfc3a10ce20
I20240426 09:49:33.770401 11106 ts_tablet_manager.cc:1909] T fddda654ba204ac4b59e1cfc3a10ce20 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:33.771098 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet fddda654ba204ac4b59e1cfc3a10ce20 (table rebalancer_dynamic_table_new_name [id=597baef885c34141ae1f6a12a3a42832]) successfully deleted
I20240426 09:49:33.782339 11107 tablet_replica.cc:330] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:33.782546 11107 raft_consensus.cc:2229] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:33.782791 11107 raft_consensus.cc:2258] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.783301 11107 ts_tablet_manager.cc:1875] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:33.784996 11107 ts_tablet_manager.cc:1888] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:33.785068 11107 log.cc:1196] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/fddda654ba204ac4b59e1cfc3a10ce20
I20240426 09:49:33.785322 11107 ts_tablet_manager.cc:1909] T fddda654ba204ac4b59e1cfc3a10ce20 P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:33.785965 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet fddda654ba204ac4b59e1cfc3a10ce20 (table rebalancer_dynamic_table_new_name [id=597baef885c34141ae1f6a12a3a42832]) successfully deleted
I20240426 09:49:33.789055 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:33.795604 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 53765d0ffcc24bb98aadc0cd0fe44af3 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.795918 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53765d0ffcc24bb98aadc0cd0fe44af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.795962 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 53765d0ffcc24bb98aadc0cd0fe44af3 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.796291 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53765d0ffcc24bb98aadc0cd0fe44af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.798250 11113 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:33.798281 11114 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:33.798427 9528 tablet_service.cc:1467] Processing CreateTablet for tablet 369c5672f4924725a7da4019241dc772 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.798630 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369c5672f4924725a7da4019241dc772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.799280 11114 tablet_bootstrap.cc:654] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.800742 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 2c41aab70e4a4b4e86cf095575cd8805 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.800908 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 53765d0ffcc24bb98aadc0cd0fe44af3 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.800917 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 2c41aab70e4a4b4e86cf095575cd8805 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.801170 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53765d0ffcc24bb98aadc0cd0fe44af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.801205 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c41aab70e4a4b4e86cf095575cd8805. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.802276 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c41aab70e4a4b4e86cf095575cd8805. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.803360 11113 tablet_bootstrap.cc:654] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.803586 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 369c5672f4924725a7da4019241dc772 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.803808 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369c5672f4924725a7da4019241dc772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.804606 11117 tablet_bootstrap.cc:492] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:33.806031 11113 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:33.806180 11113 ts_tablet_manager.cc:1367] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:33.806319 11117 tablet_bootstrap.cc:654] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.806453 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 369c5672f4924725a7da4019241dc772 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.806754 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369c5672f4924725a7da4019241dc772. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.807008 11113 raft_consensus.cc:348] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.807152 11113 raft_consensus.cc:374] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.807185 11113 raft_consensus.cc:729] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.807446 11117 tablet_bootstrap.cc:492] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:33.807518 11117 ts_tablet_manager.cc:1367] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:33.807466 11113 consensus_queue.cc:260] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.807763 11113 ts_tablet_manager.cc:1398] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.807929 11113 tablet_bootstrap.cc:492] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:33.807934 11117 raft_consensus.cc:348] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.808014 11117 raft_consensus.cc:374] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.808041 11117 raft_consensus.cc:729] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.808143 11117 consensus_queue.cc:260] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.808372 11117 ts_tablet_manager.cc:1398] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.808521 11117 tablet_bootstrap.cc:492] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:33.810025 11117 tablet_bootstrap.cc:654] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.810535 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 2c41aab70e4a4b4e86cf095575cd8805 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:33.810760 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c41aab70e4a4b4e86cf095575cd8805. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.810962 11113 tablet_bootstrap.cc:654] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.812644 11113 tablet_bootstrap.cc:492] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:33.812755 11113 ts_tablet_manager.cc:1367] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:33.813277 11113 raft_consensus.cc:348] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.813446 11113 raft_consensus.cc:374] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.813526 11113 raft_consensus.cc:729] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.813561 11117 tablet_bootstrap.cc:492] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:33.813639 11117 ts_tablet_manager.cc:1367] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20240426 09:49:33.813728 11113 consensus_queue.cc:260] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.814090 11117 raft_consensus.cc:348] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:33.814183 11117 raft_consensus.cc:374] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.814213 11117 raft_consensus.cc:729] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.814360 11116 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:33.814308 11117 consensus_queue.cc:260] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:33.814582 11113 ts_tablet_manager.cc:1398] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:33.814577 11117 ts_tablet_manager.cc:1398] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:33.815642 11116 tablet_bootstrap.cc:654] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.815788 10033 consensus_queue.cc:1035] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.816448 11124 tablet_bootstrap.cc:492] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:33.817675 11124 tablet_bootstrap.cc:654] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.819306 11124 tablet_bootstrap.cc:492] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:33.819314 11116 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:33.819396 11116 ts_tablet_manager.cc:1367] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:49:33.819447 11124 ts_tablet_manager.cc:1367] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:33.819746 8988 consensus_queue.cc:1035] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:33.819931 11124 raft_consensus.cc:348] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:33.819958 11116 raft_consensus.cc:348] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.820108 11124 raft_consensus.cc:374] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.820241 11124 raft_consensus.cc:729] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.820415 11124 consensus_queue.cc:260] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:33.820788 11124 ts_tablet_manager.cc:1398] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.820087 11116 raft_consensus.cc:374] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.821321 11116 raft_consensus.cc:729] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.821516 11114 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:33.821631 11114 ts_tablet_manager.cc:1367] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240426 09:49:33.822081 11114 raft_consensus.cc:348] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.822163 11114 raft_consensus.cc:374] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.822183 11114 raft_consensus.cc:729] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.822284 11114 consensus_queue.cc:260] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.822540 11114 ts_tablet_manager.cc:1398] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:33.821506 11116 consensus_queue.cc:260] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.822695 11114 tablet_bootstrap.cc:492] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:33.822916 11116 ts_tablet_manager.cc:1398] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:33.823590 11114 tablet_bootstrap.cc:654] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.827324 11114 tablet_bootstrap.cc:492] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:33.827392 11114 ts_tablet_manager.cc:1367] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:33.827769 11114 raft_consensus.cc:348] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:33.827850 11114 raft_consensus.cc:374] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.827883 11114 raft_consensus.cc:729] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.827986 11114 consensus_queue.cc:260] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:33.828214 11114 ts_tablet_manager.cc:1398] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.828338 11114 tablet_bootstrap.cc:492] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:33.829054 11114 tablet_bootstrap.cc:654] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.830225 11114 tablet_bootstrap.cc:492] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:33.830300 11114 ts_tablet_manager.cc:1367] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:33.830929 11114 raft_consensus.cc:348] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.831035 11114 raft_consensus.cc:374] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.831068 11114 raft_consensus.cc:729] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.831218 11114 consensus_queue.cc:260] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:33.831524 11114 ts_tablet_manager.cc:1398] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.836350 7275 consensus_queue.cc:237] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:33.837621 6880 raft_consensus.cc:1261] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:33.837622 9286 raft_consensus.cc:1261] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:33.838112 10802 consensus_queue.cc:1035] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:33.838310 11140 consensus_queue.cc:1035] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:33.840222 11143 raft_consensus.cc:2932] T a8912e37d7e941a791dd77560e2b3b6b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:33.840658 9286 raft_consensus.cc:2932] T a8912e37d7e941a791dd77560e2b3b6b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:33.840756 6880 raft_consensus.cc:2932] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:33.843478 6737 catalog_manager.cc:5484] T a8912e37d7e941a791dd77560e2b3b6b P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:33.846328 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet a8912e37d7e941a791dd77560e2b3b6b with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:33.846541 10519 tablet_replica.cc:330] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:33.846673 10519 raft_consensus.cc:2229] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:33.846858 10519 raft_consensus.cc:2258] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:33.847412 10519 ts_tablet_manager.cc:1875] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:33.849568 10519 ts_tablet_manager.cc:1888] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:33.849635 10519 log.cc:1196] T a8912e37d7e941a791dd77560e2b3b6b P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/a8912e37d7e941a791dd77560e2b3b6b
I20240426 09:49:33.850080 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet a8912e37d7e941a791dd77560e2b3b6b (table rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]) successfully deleted
I20240426 09:49:33.860296 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:33.866492 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 9829650a920940f5a2900e60676ac8fd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8769298fb0a5444391555a265b177304]), partition=
I20240426 09:49:33.866644 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 9829650a920940f5a2900e60676ac8fd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8769298fb0a5444391555a265b177304]), partition=
I20240426 09:49:33.866822 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9829650a920940f5a2900e60676ac8fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.866896 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9829650a920940f5a2900e60676ac8fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.866858 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 9829650a920940f5a2900e60676ac8fd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8769298fb0a5444391555a265b177304]), partition=
I20240426 09:49:33.867374 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9829650a920940f5a2900e60676ac8fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:33.869001 11124 tablet_bootstrap.cc:492] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:33.869554 11155 tablet_bootstrap.cc:492] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:33.869614 11117 tablet_bootstrap.cc:492] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:33.869910 11124 tablet_bootstrap.cc:654] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.870595 11117 tablet_bootstrap.cc:654] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.871343 11155 tablet_bootstrap.cc:654] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:33.871877 11117 tablet_bootstrap.cc:492] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:33.871970 11117 ts_tablet_manager.cc:1367] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:33.872438 11117 raft_consensus.cc:348] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:33.872568 11117 raft_consensus.cc:374] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.872602 11117 raft_consensus.cc:729] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.872718 11117 consensus_queue.cc:260] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:33.872826 11155 tablet_bootstrap.cc:492] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:33.873013 11155 ts_tablet_manager.cc:1367] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:33.873155 11117 ts_tablet_manager.cc:1398] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:33.873492 11155 raft_consensus.cc:348] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:33.873605 11155 raft_consensus.cc:374] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.873641 11155 raft_consensus.cc:729] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.873776 11155 consensus_queue.cc:260] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:33.874068 11155 ts_tablet_manager.cc:1398] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:33.874619 11124 tablet_bootstrap.cc:492] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:33.874725 11124 ts_tablet_manager.cc:1367] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:33.875234 11124 raft_consensus.cc:348] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:33.875362 11124 raft_consensus.cc:374] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:33.875407 11124 raft_consensus.cc:729] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:33.875537 11124 consensus_queue.cc:260] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:33.875842 11124 ts_tablet_manager.cc:1398] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:33.902846 11143 raft_consensus.cc:482] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:33.902972 11143 raft_consensus.cc:504] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:33.903421 11143 leader_election.cc:288] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:33.903999 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9829650a920940f5a2900e60676ac8fd" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:33.904171 9286 raft_consensus.cc:2454] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:33.904461 7209 leader_election.cc:302] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:33.904625 11140 raft_consensus.cc:2781] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:33.904680 11140 raft_consensus.cc:482] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:33.904721 11140 raft_consensus.cc:3037] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.905687 11140 raft_consensus.cc:504] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:33.906064 11140 leader_election.cc:288] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:33.906642 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9829650a920940f5a2900e60676ac8fd" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:33.906751 9286 raft_consensus.cc:3037] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.907639 10164 raft_consensus.cc:482] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:33.907760 9286 raft_consensus.cc:2454] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:33.907733 10164 raft_consensus.cc:504] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.908123 10164 leader_election.cc:288] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:33.908699 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53765d0ffcc24bb98aadc0cd0fe44af3" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:33.908825 9549 raft_consensus.cc:2454] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:33.909282 9614 leader_election.cc:302] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:33.910074 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53765d0ffcc24bb98aadc0cd0fe44af3" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:33.910224 6880 raft_consensus.cc:2454] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:33.910878 7209 leader_election.cc:302] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:33.911535 10164 raft_consensus.cc:2781] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:33.911605 10164 raft_consensus.cc:482] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:33.911657 10164 raft_consensus.cc:3037] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.912606 10164 raft_consensus.cc:504] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.912894 10164 leader_election.cc:288] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:33.913208 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53765d0ffcc24bb98aadc0cd0fe44af3" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:33.913255 11140 raft_consensus.cc:2781] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:33.913318 6880 raft_consensus.cc:3037] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.913337 11140 raft_consensus.cc:686] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:33.913475 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9829650a920940f5a2900e60676ac8fd" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:33.913468 11140 consensus_queue.cc:237] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:33.913587 9416 raft_consensus.cc:2454] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:33.913782 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9829650a920940f5a2900e60676ac8fd" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:33.913868 9417 raft_consensus.cc:3037] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.914918 6880 raft_consensus.cc:2454] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:33.914930 9417 raft_consensus.cc:2454] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:33.915266 9616 leader_election.cc:302] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:33.915431 10164 raft_consensus.cc:2781] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:33.915522 10164 raft_consensus.cc:686] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:33.915637 10164 consensus_queue.cc:237] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:33.915853 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53765d0ffcc24bb98aadc0cd0fe44af3" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:33.915988 9549 raft_consensus.cc:3037] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.916712 6733 catalog_manager.cc:5484] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:33.916963 9549 raft_consensus.cc:2454] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:33.917687 6740 catalog_manager.cc:5484] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:33.977437 9917 raft_consensus.cc:482] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:33.977739 9917 raft_consensus.cc:504] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:33.978261 9917 leader_election.cc:288] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:33.978636 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369c5672f4924725a7da4019241dc772" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:33.978806 9417 raft_consensus.cc:2454] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:33.978828 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369c5672f4924725a7da4019241dc772" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:33.978977 9286 raft_consensus.cc:2454] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:33.979203 9482 leader_election.cc:302] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:33.979413 9917 raft_consensus.cc:2781] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:33.979457 9917 raft_consensus.cc:482] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:33.979492 9917 raft_consensus.cc:3037] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.980285 9917 raft_consensus.cc:504] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:33.980621 9917 leader_election.cc:288] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:33.980913 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369c5672f4924725a7da4019241dc772" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:33.981034 9286 raft_consensus.cc:3037] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.981034 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369c5672f4924725a7da4019241dc772" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:33.981129 9417 raft_consensus.cc:3037] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:33.981997 9286 raft_consensus.cc:2454] T 369c5672f4924725a7da4019241dc772 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:33.982060 9417 raft_consensus.cc:2454] T 369c5672f4924725a7da4019241dc772 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:33.982321 9483 leader_election.cc:302] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:33.982546 9917 raft_consensus.cc:2781] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:33.982617 9917 raft_consensus.cc:686] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:33.982721 9917 consensus_queue.cc:237] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:33.984377 6734 catalog_manager.cc:5484] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.004351 10201 raft_consensus.cc:482] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:34.004456 10201 raft_consensus.cc:504] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.004770 10201 leader_election.cc:288] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:34.005136 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c41aab70e4a4b4e86cf095575cd8805" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:34.005162 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c41aab70e4a4b4e86cf095575cd8805" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:34.005251 9286 raft_consensus.cc:2454] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:34.005288 9549 raft_consensus.cc:2454] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:34.005502 9615 leader_election.cc:302] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:34.005657 10201 raft_consensus.cc:2781] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:34.005718 10201 raft_consensus.cc:482] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:34.005774 10201 raft_consensus.cc:3037] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.006539 10201 raft_consensus.cc:504] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.006858 10201 leader_election.cc:288] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:34.007088 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c41aab70e4a4b4e86cf095575cd8805" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:34.007161 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c41aab70e4a4b4e86cf095575cd8805" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:34.007187 9286 raft_consensus.cc:3037] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.007262 9549 raft_consensus.cc:3037] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.007905 9286 raft_consensus.cc:2454] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:34.007962 9549 raft_consensus.cc:2454] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:34.008181 9615 leader_election.cc:302] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:34.008374 10201 raft_consensus.cc:2781] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:34.008448 10201 raft_consensus.cc:686] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:34.008566 10201 consensus_queue.cc:237] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.010321 6734 catalog_manager.cc:5484] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:34.021770 6734 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:34.025465 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:34.028066 7123 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:34.033221 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:34.034672 9286 raft_consensus.cc:1261] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:34.035341 11140 consensus_queue.cc:1035] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.035979 6862 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:34.038121 6740 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240426 09:49:34.038246 6740 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240426 09:49:34.039541 6991 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:34.039566 10960 tablet.cc:1722] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:34.039885 10960 tablet.cc:1620] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:34.040409 9417 raft_consensus.cc:1261] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:34.040542 11206 tablet.cc:1722] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:34.040926 11206 tablet.cc:1620] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:34.041144 11143 consensus_queue.cc:1035] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.042524 6734 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet 9829650a920940f5a2900e60676ac8fd (table rebalancer_dynamic_table [id=8769298fb0a5444391555a265b177304]). Expected version 1 got 0
I20240426 09:49:34.044576 10731 tablet.cc:1722] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:34.044845 10731 tablet.cc:1620] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:34.045125 9529 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:34.047060 9661 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:34.048556 6732 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 9829650a920940f5a2900e60676ac8fd (table rebalancer_dynamic_table [id=8769298fb0a5444391555a265b177304]). Expected version 1 got 0
I20240426 09:49:34.055626 6740 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 6dc287ff7ea640a38537c74e9ff156f7
I20240426 09:49:34.055923 6740 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 9ed2745f48d141c9a89ae8e13606a782
I20240426 09:49:34.056092 6740 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 4 replicas of tablet b85b6cfec5314d75aebe17670edf8d04
I20240426 09:49:34.057119 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet b85b6cfec5314d75aebe17670edf8d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:34.057289 10858 tablet_replica.cc:330] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:34.057406 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ed2745f48d141c9a89ae8e13606a782 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:34.057416 10858 raft_consensus.cc:2229] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.057544 10858 raft_consensus.cc:2258] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.057546 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet b85b6cfec5314d75aebe17670edf8d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:34.057900 10519 tablet_replica.cc:330] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:34.058019 10519 raft_consensus.cc:2229] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.058035 10858 ts_tablet_manager.cc:1875] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.058145 10519 raft_consensus.cc:2258] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.058571 10519 ts_tablet_manager.cc:1875] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.060173 10858 ts_tablet_manager.cc:1888] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:34.060235 10858 log.cc:1196] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/b85b6cfec5314d75aebe17670edf8d04
I20240426 09:49:34.060755 10519 ts_tablet_manager.cc:1888] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.060722 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ed2745f48d141c9a89ae8e13606a782 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:34.060815 10519 log.cc:1196] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/9ed2745f48d141c9a89ae8e13606a782
I20240426 09:49:34.061059 10519 ts_tablet_manager.cc:1909] T 9ed2745f48d141c9a89ae8e13606a782 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:34.060547 10858 ts_tablet_manager.cc:1909] T b85b6cfec5314d75aebe17670edf8d04 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:34.061717 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 9ed2745f48d141c9a89ae8e13606a782 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.062009 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dc287ff7ea640a38537c74e9ff156f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:34.062660 11107 tablet_replica.cc:330] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:34.062796 10519 tablet_replica.cc:330] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:34.062811 11107 raft_consensus.cc:2229] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.062901 10519 raft_consensus.cc:2229] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.062932 11107 raft_consensus.cc:2258] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.063020 10519 raft_consensus.cc:2258] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.063465 10519 ts_tablet_manager.cc:1875] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.063894 11107 ts_tablet_manager.cc:1875] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.065150 6861 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ed2745f48d141c9a89ae8e13606a782 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:34.065320 6860 tablet_service.cc:1514] Processing DeleteTablet for tablet b85b6cfec5314d75aebe17670edf8d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:34.065374 11030 tablet_replica.cc:330] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:34.065531 11030 raft_consensus.cc:2229] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:34.065755 11030 raft_consensus.cc:2258] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.065928 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dc287ff7ea640a38537c74e9ff156f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:34.066015 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet b85b6cfec5314d75aebe17670edf8d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:34.066099 10689 tablet_replica.cc:330] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:34.066097 11107 ts_tablet_manager.cc:1888] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240426 09:49:34.066156 11107 log.cc:1196] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/6dc287ff7ea640a38537c74e9ff156f7
I20240426 09:49:34.066232 10689 raft_consensus.cc:2229] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.066314 11030 ts_tablet_manager.cc:1875] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.066437 11107 ts_tablet_manager.cc:1909] T 6dc287ff7ea640a38537c74e9ff156f7 P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:34.066429 10689 raft_consensus.cc:2258] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.067023 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dc287ff7ea640a38537c74e9ff156f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:34.067656 10689 ts_tablet_manager.cc:1875] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.067677 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 6dc287ff7ea640a38537c74e9ff156f7 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.069079 11030 ts_tablet_manager.cc:1888] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.069145 11030 log.cc:1196] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/9ed2745f48d141c9a89ae8e13606a782
I20240426 09:49:34.069418 11030 ts_tablet_manager.cc:1909] T 9ed2745f48d141c9a89ae8e13606a782 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:34.069774 10689 ts_tablet_manager.cc:1888] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240426 09:49:34.069836 10689 log.cc:1196] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/6dc287ff7ea640a38537c74e9ff156f7
I20240426 09:49:34.070097 10689 ts_tablet_manager.cc:1909] T 6dc287ff7ea640a38537c74e9ff156f7 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:34.071043 11030 tablet_replica.cc:330] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:34.071184 11030 raft_consensus.cc:2229] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.071189 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 9ed2745f48d141c9a89ae8e13606a782 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.071346 11030 raft_consensus.cc:2258] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.072057 11030 ts_tablet_manager.cc:1875] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.073029 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=8769298fb0a5444391555a265b177304] alter complete (version 1)
I20240426 09:49:34.073954 10689 tablet_replica.cc:330] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:34.074141 11030 ts_tablet_manager.cc:1888] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:34.074252 11030 log.cc:1196] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/b85b6cfec5314d75aebe17670edf8d04
I20240426 09:49:34.074203 10689 raft_consensus.cc:2229] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:34.074477 10689 raft_consensus.cc:2258] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.074563 11030 ts_tablet_manager.cc:1909] T b85b6cfec5314d75aebe17670edf8d04 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:34.074976 10689 ts_tablet_manager.cc:1875] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.075134 11030 tablet_replica.cc:330] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:34.075274 11030 raft_consensus.cc:2229] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:34.075372 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet b85b6cfec5314d75aebe17670edf8d04 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.075461 11030 raft_consensus.cc:2258] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.076061 11030 ts_tablet_manager.cc:1875] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.077713 11030 ts_tablet_manager.cc:1888] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240426 09:49:34.077813 11030 log.cc:1196] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/6dc287ff7ea640a38537c74e9ff156f7
I20240426 09:49:34.078101 11030 ts_tablet_manager.cc:1909] T 6dc287ff7ea640a38537c74e9ff156f7 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:34.078859 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 6dc287ff7ea640a38537c74e9ff156f7 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.079560 10519 ts_tablet_manager.cc:1888] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:34.079869 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 6dc287ff7ea640a38537c74e9ff156f7 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.080050 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet b85b6cfec5314d75aebe17670edf8d04 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.080243 10689 ts_tablet_manager.cc:1888] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:34.080329 10689 log.cc:1196] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/b85b6cfec5314d75aebe17670edf8d04
I20240426 09:49:34.080591 10689 ts_tablet_manager.cc:1909] T b85b6cfec5314d75aebe17670edf8d04 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:34.081118 10519 log.cc:1196] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/b85b6cfec5314d75aebe17670edf8d04
I20240426 09:49:34.081166 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet b85b6cfec5314d75aebe17670edf8d04 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.081382 10519 ts_tablet_manager.cc:1909] T b85b6cfec5314d75aebe17670edf8d04 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:34.082019 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet b85b6cfec5314d75aebe17670edf8d04 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.091550 7255 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:34.099020 11208 tablet_replica.cc:330] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:34.099196 11208 raft_consensus.cc:2229] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.099359 11208 raft_consensus.cc:2258] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.099925 11208 ts_tablet_manager.cc:1875] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.103983 11208 ts_tablet_manager.cc:1888] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.104135 11208 log.cc:1196] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/9ed2745f48d141c9a89ae8e13606a782
I20240426 09:49:34.104463 11208 ts_tablet_manager.cc:1909] T 9ed2745f48d141c9a89ae8e13606a782 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:34.105131 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 9ed2745f48d141c9a89ae8e13606a782 (table rebalance_test_table_14 [id=78f905d81fea48f0843b3bba99c5dced]) successfully deleted
I20240426 09:49:34.130208 7011 consensus_queue.cc:237] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.131587 9416 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:34.131938 7142 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:34.132010 11210 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:34.132313 8988 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:34.134174 11210 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.134960 7142 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.134548 9416 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.138809 6740 catalog_manager.cc:5484] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.144405 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c94c820ab65420a8d3f885ccc9f8884 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:34.144704 11208 tablet_replica.cc:330] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:34.144843 11208 raft_consensus.cc:2229] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.144958 11208 raft_consensus.cc:2258] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.145516 11208 ts_tablet_manager.cc:1875] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:34.147562 11208 ts_tablet_manager.cc:1888] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:34.147946 11208 log.cc:1196] T 9c94c820ab65420a8d3f885ccc9f8884 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/9c94c820ab65420a8d3f885ccc9f8884
I20240426 09:49:34.148548 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 9c94c820ab65420a8d3f885ccc9f8884 (table rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]) successfully deleted
I20240426 09:49:34.180678 6734 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:34.186861 11206 tablet.cc:1722] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:34.187022 10960 tablet.cc:1722] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:34.187179 11206 tablet.cc:1620] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:34.187276 10960 tablet.cc:1620] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:34.189697 10731 tablet.cc:1722] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:34.189990 10731 tablet.cc:1620] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:34.193867 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=8769298fb0a5444391555a265b177304] alter complete (version 2)
I20240426 09:49:34.239917 6993 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:34.240729 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:34.241027 7122 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:34.241653 9661 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:34.247921 9529 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:34.249326 6862 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:34.250983 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:34.251506 7255 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:34.297374 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:34.303341 11206 tablet.cc:1722] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:34.304060 10960 tablet.cc:1722] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:34.304056 10731 tablet.cc:1722] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:34.308799 6736 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=8769298fb0a5444391555a265b177304] alter complete (version 3)
I20240426 09:49:34.319692 6880 consensus_queue.cc:237] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.321192 9672 raft_consensus.cc:1261] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:34.321419 7275 raft_consensus.cc:1261] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:34.321549 11296 consensus_queue.cc:1035] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:34.321782 11211 consensus_queue.cc:1035] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:34.323683 11211 raft_consensus.cc:2932] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:34.325388 9672 raft_consensus.cc:2932] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:34.326457 6733 catalog_manager.cc:5484] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:34.327683 7275 raft_consensus.cc:2932] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:34.331073 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2fd1740d544f5d98741aecb64cdfc7 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:34.331279 10519 tablet_replica.cc:330] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:34.331418 10519 raft_consensus.cc:2229] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.331588 10519 raft_consensus.cc:2258] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.332084 10519 ts_tablet_manager.cc:1875] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:34.333897 10519 ts_tablet_manager.cc:1888] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:34.333949 10519 log.cc:1196] T 1c2fd1740d544f5d98741aecb64cdfc7 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/1c2fd1740d544f5d98741aecb64cdfc7
I20240426 09:49:34.334412 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 1c2fd1740d544f5d98741aecb64cdfc7 (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:34.379484 10164 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.382731 10164 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.411026 6734 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:34.411221 6734 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:34.413604 6734 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 9829650a920940f5a2900e60676ac8fd
I20240426 09:49:34.414443 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 9829650a920940f5a2900e60676ac8fd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:34.414619 11208 tablet_replica.cc:330] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:34.414772 11208 raft_consensus.cc:2229] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:34.415019 11208 raft_consensus.cc:2258] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.415772 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 9829650a920940f5a2900e60676ac8fd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:34.416564 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 9829650a920940f5a2900e60676ac8fd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:34.416744 11107 tablet_replica.cc:330] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:34.416898 11107 raft_consensus.cc:2229] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.417034 11107 raft_consensus.cc:2258] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.417727 11107 ts_tablet_manager.cc:1875] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.417802 11208 ts_tablet_manager.cc:1875] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.418161 6993 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:34.419872 11208 ts_tablet_manager.cc:1888] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:34.419872 11107 ts_tablet_manager.cc:1888] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:34.419945 11107 log.cc:1196] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/9829650a920940f5a2900e60676ac8fd
I20240426 09:49:34.419945 11208 log.cc:1196] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/9829650a920940f5a2900e60676ac8fd
I20240426 09:49:34.420197 11107 ts_tablet_manager.cc:1909] T 9829650a920940f5a2900e60676ac8fd P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:34.422740 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 9829650a920940f5a2900e60676ac8fd (table rebalancer_dynamic_table_new_name [id=8769298fb0a5444391555a265b177304]) successfully deleted
I20240426 09:49:34.422878 9661 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:34.422945 11208 ts_tablet_manager.cc:1909] T 9829650a920940f5a2900e60676ac8fd P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:34.424031 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 9829650a920940f5a2900e60676ac8fd (table rebalancer_dynamic_table_new_name [id=8769298fb0a5444391555a265b177304]) successfully deleted
I20240426 09:49:34.426944 6862 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:34.428786 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:34.429388 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:34.429730 9917 consensus_queue.cc:1035] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.442077 11354 tablet_replica.cc:330] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:34.442555 11354 raft_consensus.cc:2229] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.443110 11354 raft_consensus.cc:2258] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.443918 11354 ts_tablet_manager.cc:1875] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.445343 10164 consensus_queue.cc:1035] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.447057 11354 ts_tablet_manager.cc:1888] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:34.447180 11354 log.cc:1196] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/9829650a920940f5a2900e60676ac8fd
I20240426 09:49:34.447490 11354 ts_tablet_manager.cc:1909] T 9829650a920940f5a2900e60676ac8fd P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:34.448138 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 9829650a920940f5a2900e60676ac8fd (table rebalancer_dynamic_table_new_name [id=8769298fb0a5444391555a265b177304]) successfully deleted
I20240426 09:49:34.449961 7122 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:34.450626 9529 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:34.456966 10912 consensus_queue.cc:1035] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.459595 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:34.472615 10201 consensus_queue.cc:1035] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.508266 7142 consensus_queue.cc:237] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.509593 7011 raft_consensus.cc:1261] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:34.509964 9416 raft_consensus.cc:1261] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:34.509960 9907 consensus_queue.cc:1035] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:34.510455 9907 consensus_queue.cc:1035] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:34.512290 9907 raft_consensus.cc:2932] T 0f67d94463ce46708246808da0b100e9 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.512573 7011 raft_consensus.cc:2932] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.514624 9416 raft_consensus.cc:2932] T 0f67d94463ce46708246808da0b100e9 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.517494 6731 catalog_manager.cc:5484] T 0f67d94463ce46708246808da0b100e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.519135 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:34.521636 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f67d94463ce46708246808da0b100e9 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:34.521867 11208 tablet_replica.cc:330] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:34.521979 11208 raft_consensus.cc:2229] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.522117 11208 raft_consensus.cc:2258] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.522617 11208 ts_tablet_manager.cc:1875] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:34.524845 11208 ts_tablet_manager.cc:1888] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:34.524919 11208 log.cc:1196] T 0f67d94463ce46708246808da0b100e9 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/0f67d94463ce46708246808da0b100e9
I20240426 09:49:34.525043 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 1c210fb4ffdf429ab95be6b01603ecc5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=93eb47e7e91041e0b31600c7f9f23551]), partition=
I20240426 09:49:34.525224 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 1c210fb4ffdf429ab95be6b01603ecc5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=93eb47e7e91041e0b31600c7f9f23551]), partition=
I20240426 09:49:34.525323 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c210fb4ffdf429ab95be6b01603ecc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.525434 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c210fb4ffdf429ab95be6b01603ecc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.527679 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 1c210fb4ffdf429ab95be6b01603ecc5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=93eb47e7e91041e0b31600c7f9f23551]), partition=
I20240426 09:49:34.527932 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c210fb4ffdf429ab95be6b01603ecc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.530608 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 0f67d94463ce46708246808da0b100e9 (table rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]) successfully deleted
I20240426 09:49:34.530985 11385 tablet_bootstrap.cc:492] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:34.532096 11384 tablet_bootstrap.cc:492] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:34.532783 11385 tablet_bootstrap.cc:654] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.534216 11384 tablet_bootstrap.cc:654] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.534955 11386 tablet_bootstrap.cc:492] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:34.536092 11386 tablet_bootstrap.cc:654] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.536222 11385 tablet_bootstrap.cc:492] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:34.536370 11385 ts_tablet_manager.cc:1367] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:34.536942 11385 raft_consensus.cc:348] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:34.537166 11386 tablet_bootstrap.cc:492] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:34.537181 11385 raft_consensus.cc:374] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.537228 11385 raft_consensus.cc:729] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.537243 11386 ts_tablet_manager.cc:1367] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:34.537364 11385 consensus_queue.cc:260] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:34.537647 11386 raft_consensus.cc:348] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:34.537739 11386 raft_consensus.cc:374] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.537752 11385 ts_tablet_manager.cc:1398] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:34.537766 11386 raft_consensus.cc:729] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.537880 11386 consensus_queue.cc:260] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:34.538110 11386 ts_tablet_manager.cc:1398] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:34.538425 11384 tablet_bootstrap.cc:492] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:34.538504 11384 ts_tablet_manager.cc:1367] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:34.539369 11384 raft_consensus.cc:348] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:34.539489 11384 raft_consensus.cc:374] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.539728 11384 raft_consensus.cc:729] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.539906 11384 consensus_queue.cc:260] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:34.540282 11384 ts_tablet_manager.cc:1398] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:34.551599 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:34.557735 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 8cb2e6c9a044463aa58878e06fc3fff2 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:34.558035 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb2e6c9a044463aa58878e06fc3fff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.558218 9266 tablet_service.cc:1467] Processing CreateTablet for tablet d0779912d7a04b4c95dd1db6f9c4d6fd (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:34.558343 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 8cb2e6c9a044463aa58878e06fc3fff2 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:34.558455 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0779912d7a04b4c95dd1db6f9c4d6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.558578 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb2e6c9a044463aa58878e06fc3fff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.559157 9528 tablet_service.cc:1467] Processing CreateTablet for tablet cd67a40e0b834a83b41259ee4da3f3db (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:34.559358 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd67a40e0b834a83b41259ee4da3f3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.561208 11385 tablet_bootstrap.cc:492] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:34.561548 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 8cb2e6c9a044463aa58878e06fc3fff2 (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:34.561816 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb2e6c9a044463aa58878e06fc3fff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.562165 11385 tablet_bootstrap.cc:654] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.562825 11384 tablet_bootstrap.cc:492] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:34.563273 11385 tablet_bootstrap.cc:492] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:34.563340 11385 ts_tablet_manager.cc:1367] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:34.563710 11384 tablet_bootstrap.cc:654] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.563881 11385 raft_consensus.cc:348] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.563977 11385 raft_consensus.cc:374] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.564003 11385 raft_consensus.cc:729] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.564124 11385 consensus_queue.cc:260] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.564375 11385 ts_tablet_manager.cc:1398] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:34.564622 9661 tablet_service.cc:1467] Processing CreateTablet for tablet d0779912d7a04b4c95dd1db6f9c4d6fd (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:34.564898 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0779912d7a04b4c95dd1db6f9c4d6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.567029 9660 tablet_service.cc:1467] Processing CreateTablet for tablet cd67a40e0b834a83b41259ee4da3f3db (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:34.567250 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd67a40e0b834a83b41259ee4da3f3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.567606 6739 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240426 09:49:34.567723 6739 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240426 09:49:34.568274 11384 tablet_bootstrap.cc:492] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:34.568356 11384 ts_tablet_manager.cc:1367] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:34.568796 11384 raft_consensus.cc:348] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.568918 11384 raft_consensus.cc:374] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.568949 11384 raft_consensus.cc:729] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.569053 11384 consensus_queue.cc:260] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.569335 11384 ts_tablet_manager.cc:1398] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:34.569489 11384 tablet_bootstrap.cc:492] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:34.569932 7122 tablet_service.cc:1467] Processing CreateTablet for tablet d0779912d7a04b4c95dd1db6f9c4d6fd (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:34.570209 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0779912d7a04b4c95dd1db6f9c4d6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.570370 11384 tablet_bootstrap.cc:654] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.572703 7123 tablet_service.cc:1467] Processing CreateTablet for tablet cd67a40e0b834a83b41259ee4da3f3db (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:34.572947 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd67a40e0b834a83b41259ee4da3f3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:34.574870 11386 tablet_bootstrap.cc:492] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:34.574963 6739 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 30206c7050b64ad2900c789f48d3390f
I20240426 09:49:34.575469 11385 tablet_bootstrap.cc:492] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:34.575443 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 30206c7050b64ad2900c789f48d3390f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:34.575631 11208 tablet_replica.cc:330] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:34.575819 11208 raft_consensus.cc:2229] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.575841 11386 tablet_bootstrap.cc:654] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.575966 11208 raft_consensus.cc:2258] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.576512 11208 ts_tablet_manager.cc:1875] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.576560 11385 tablet_bootstrap.cc:654] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.577459 11405 tablet_bootstrap.cc:492] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:34.577713 11397 tablet_bootstrap.cc:492] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:34.578573 11208 ts_tablet_manager.cc:1888] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.579157 11208 log.cc:1196] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/30206c7050b64ad2900c789f48d3390f
I20240426 09:49:34.579423 11208 ts_tablet_manager.cc:1909] T 30206c7050b64ad2900c789f48d3390f P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:34.579485 11397 tablet_bootstrap.cc:654] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.580042 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 30206c7050b64ad2900c789f48d3390f (table rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]) successfully deleted
I20240426 09:49:34.580806 11405 tablet_bootstrap.cc:654] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.581044 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 30206c7050b64ad2900c789f48d3390f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:34.581280 10689 tablet_replica.cc:330] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:34.581430 10689 raft_consensus.cc:2229] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:34.581609 10689 raft_consensus.cc:2258] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.582074 11384 tablet_bootstrap.cc:492] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:34.582085 10689 ts_tablet_manager.cc:1875] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.582155 11384 ts_tablet_manager.cc:1367] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:34.582655 11384 raft_consensus.cc:348] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.582818 11384 raft_consensus.cc:374] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.582850 11384 raft_consensus.cc:729] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.582981 11384 consensus_queue.cc:260] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.583334 11384 ts_tablet_manager.cc:1398] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:34.583891 11385 tablet_bootstrap.cc:492] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:34.583997 11385 ts_tablet_manager.cc:1367] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:34.584256 10689 ts_tablet_manager.cc:1888] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.584316 10689 log.cc:1196] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/30206c7050b64ad2900c789f48d3390f
I20240426 09:49:34.584420 11386 tablet_bootstrap.cc:492] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:34.584434 11385 raft_consensus.cc:348] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.584502 11386 ts_tablet_manager.cc:1367] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:34.584537 11385 raft_consensus.cc:374] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.584559 10689 ts_tablet_manager.cc:1909] T 30206c7050b64ad2900c789f48d3390f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:34.584568 11385 raft_consensus.cc:729] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.584702 11385 consensus_queue.cc:260] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.584966 11385 ts_tablet_manager.cc:1398] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:34.584964 11386 raft_consensus.cc:348] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.585062 11386 raft_consensus.cc:374] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.585095 11386 raft_consensus.cc:729] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.585213 11386 consensus_queue.cc:260] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.585481 11386 ts_tablet_manager.cc:1398] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:34.585584 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 30206c7050b64ad2900c789f48d3390f (table rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]) successfully deleted
I20240426 09:49:34.585649 11386 tablet_bootstrap.cc:492] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:34.585896 11397 tablet_bootstrap.cc:492] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:34.585997 11397 ts_tablet_manager.cc:1367] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:34.586448 11386 tablet_bootstrap.cc:654] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.587116 11397 raft_consensus.cc:348] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.587285 6739 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 6111d5f14ab5490692a69d193ae4aa45
I20240426 09:49:34.587407 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 30206c7050b64ad2900c789f48d3390f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:34.587761 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 6111d5f14ab5490692a69d193ae4aa45 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:34.587922 10519 tablet_replica.cc:330] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:34.588048 10519 raft_consensus.cc:2229] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.588228 6739 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 154f16534c294c99b1fcb1735b838d66
I20240426 09:49:34.588250 10519 raft_consensus.cc:2258] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.588516 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 6111d5f14ab5490692a69d193ae4aa45 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:34.588603 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 154f16534c294c99b1fcb1735b838d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:34.588762 11208 tablet_replica.cc:330] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:34.588739 10519 ts_tablet_manager.cc:1875] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.588874 11208 raft_consensus.cc:2229] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:34.589056 11208 raft_consensus.cc:2258] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.589524 11208 ts_tablet_manager.cc:1875] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.587309 11397 raft_consensus.cc:374] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.589823 11397 raft_consensus.cc:729] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.590061 11397 consensus_queue.cc:260] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.591060 10519 ts_tablet_manager.cc:1888] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.591133 10519 log.cc:1196] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/6111d5f14ab5490692a69d193ae4aa45
I20240426 09:49:34.591413 10519 ts_tablet_manager.cc:1909] T 6111d5f14ab5490692a69d193ae4aa45 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:34.591575 11208 ts_tablet_manager.cc:1888] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.591630 11208 log.cc:1196] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/154f16534c294c99b1fcb1735b838d66
I20240426 09:49:34.592080 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 6111d5f14ab5490692a69d193ae4aa45 (table rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]) successfully deleted
I20240426 09:49:34.591933 11208 ts_tablet_manager.cc:1909] T 154f16534c294c99b1fcb1735b838d66 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:34.592657 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 6111d5f14ab5490692a69d193ae4aa45 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:34.592785 11386 tablet_bootstrap.cc:492] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:34.592842 10689 tablet_replica.cc:330] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:34.592877 11386 ts_tablet_manager.cc:1367] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:34.592880 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 154f16534c294c99b1fcb1735b838d66 (table rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]) successfully deleted
I20240426 09:49:34.592957 10689 raft_consensus.cc:2229] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:34.593156 10689 raft_consensus.cc:2258] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.593355 11386 raft_consensus.cc:348] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.593423 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 154f16534c294c99b1fcb1735b838d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:34.593451 11386 raft_consensus.cc:374] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.593488 11386 raft_consensus.cc:729] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.593572 10519 tablet_replica.cc:330] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:34.593600 11386 consensus_queue.cc:260] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.593680 10519 raft_consensus.cc:2229] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.593792 10519 raft_consensus.cc:2258] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.593806 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 154f16534c294c99b1fcb1735b838d66 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:34 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:34.593892 11386 ts_tablet_manager.cc:1398] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:34.594229 10519 ts_tablet_manager.cc:1875] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.595000 10689 ts_tablet_manager.cc:1875] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.596572 11397 ts_tablet_manager.cc:1398] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20240426 09:49:34.597925 11405 tablet_bootstrap.cc:492] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:34.598011 11405 ts_tablet_manager.cc:1367] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20240426 09:49:34.598243 10689 ts_tablet_manager.cc:1888] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.598318 10689 log.cc:1196] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/6111d5f14ab5490692a69d193ae4aa45
I20240426 09:49:34.598379 11405 raft_consensus.cc:348] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.598467 11405 raft_consensus.cc:374] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.598520 11405 raft_consensus.cc:729] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.598568 10689 ts_tablet_manager.cc:1909] T 6111d5f14ab5490692a69d193ae4aa45 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:34.598639 11405 consensus_queue.cc:260] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.599406 11414 tablet_replica.cc:330] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:34.599541 10519 ts_tablet_manager.cc:1888] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.599609 10519 log.cc:1196] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/154f16534c294c99b1fcb1735b838d66
I20240426 09:49:34.599706 11405 ts_tablet_manager.cc:1398] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:34.599846 11405 tablet_bootstrap.cc:492] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:34.600628 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 6111d5f14ab5490692a69d193ae4aa45 (table rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]) successfully deleted
I20240426 09:49:34.600971 11405 tablet_bootstrap.cc:654] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:34.601362 11414 raft_consensus.cc:2229] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.599862 10519 ts_tablet_manager.cc:1909] T 154f16534c294c99b1fcb1735b838d66 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:34.602547 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 154f16534c294c99b1fcb1735b838d66 (table rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]) successfully deleted
I20240426 09:49:34.602587 11405 tablet_bootstrap.cc:492] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:34.602649 11405 ts_tablet_manager.cc:1367] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240426 09:49:34.602797 11414 raft_consensus.cc:2258] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.603199 11405 raft_consensus.cc:348] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.603333 11405 raft_consensus.cc:374] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:34.603406 11405 raft_consensus.cc:729] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:34.603452 11414 ts_tablet_manager.cc:1875] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.603529 11405 consensus_queue.cc:260] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.604395 11405 ts_tablet_manager.cc:1398] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:34.605988 11414 ts_tablet_manager.cc:1888] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.606074 11414 log.cc:1196] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/30206c7050b64ad2900c789f48d3390f
I20240426 09:49:34.606410 11414 ts_tablet_manager.cc:1909] T 30206c7050b64ad2900c789f48d3390f P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:34.607059 11414 tablet_replica.cc:330] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:34.607218 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 30206c7050b64ad2900c789f48d3390f (table rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]) successfully deleted
I20240426 09:49:34.607234 11414 raft_consensus.cc:2229] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.607412 11414 raft_consensus.cc:2258] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.607908 11414 ts_tablet_manager.cc:1875] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.610215 11414 ts_tablet_manager.cc:1888] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.610334 11414 log.cc:1196] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/6111d5f14ab5490692a69d193ae4aa45
I20240426 09:49:34.610734 11414 ts_tablet_manager.cc:1909] T 6111d5f14ab5490692a69d193ae4aa45 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:34.611346 11414 tablet_replica.cc:330] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:34.611442 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 6111d5f14ab5490692a69d193ae4aa45 (table rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]) successfully deleted
I20240426 09:49:34.611572 11414 raft_consensus.cc:2229] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:34.611786 11414 raft_consensus.cc:2258] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:34.612250 11414 ts_tablet_manager.cc:1875] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:34.614101 11414 ts_tablet_manager.cc:1888] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:34.614221 11414 log.cc:1196] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/154f16534c294c99b1fcb1735b838d66
I20240426 09:49:34.614537 11414 ts_tablet_manager.cc:1909] T 154f16534c294c99b1fcb1735b838d66 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:34.615965 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 154f16534c294c99b1fcb1735b838d66 (table rebalance_test_table_16 [id=b2bc13b67ec7455d9fbb389464227578]) successfully deleted
I20240426 09:49:34.642309 10912 raft_consensus.cc:482] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:34.642464 10912 raft_consensus.cc:504] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:34.642980 10912 leader_election.cc:288] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:34.643376 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c210fb4ffdf429ab95be6b01603ecc5" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:34.643376 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c210fb4ffdf429ab95be6b01603ecc5" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:34.643533 9285 raft_consensus.cc:2454] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:34.643533 9672 raft_consensus.cc:2454] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:34.643810 9483 leader_election.cc:302] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:34.644006 10912 raft_consensus.cc:2781] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:34.644070 10912 raft_consensus.cc:482] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:34.644126 10912 raft_consensus.cc:3037] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.645220 10912 raft_consensus.cc:504] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:34.645603 10912 leader_election.cc:288] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:34.645879 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c210fb4ffdf429ab95be6b01603ecc5" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:34.645998 9285 raft_consensus.cc:3037] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.645998 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c210fb4ffdf429ab95be6b01603ecc5" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:34.646099 9672 raft_consensus.cc:3037] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.646860 9285 raft_consensus.cc:2454] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:34.647007 9672 raft_consensus.cc:2454] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:34.647186 9483 leader_election.cc:302] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:34.647451 10912 raft_consensus.cc:2781] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:34.647532 10912 raft_consensus.cc:686] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:34.647668 10912 consensus_queue.cc:237] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:34.649840 6737 catalog_manager.cc:5484] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:34.706888 10894 raft_consensus.cc:482] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:34.707021 10894 raft_consensus.cc:504] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.707532 10894 leader_election.cc:288] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:34.707880 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0779912d7a04b4c95dd1db6f9c4d6fd" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:34.708032 9285 raft_consensus.cc:2454] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:34.708355 7078 leader_election.cc:302] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:34.708577 10894 raft_consensus.cc:2781] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:34.708631 10894 raft_consensus.cc:482] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:34.708657 10894 raft_consensus.cc:3037] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.709472 10894 raft_consensus.cc:504] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.709787 10894 leader_election.cc:288] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:34.710069 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0779912d7a04b4c95dd1db6f9c4d6fd" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:34.710202 9285 raft_consensus.cc:3037] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.711014 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0779912d7a04b4c95dd1db6f9c4d6fd" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:34.711162 9672 raft_consensus.cc:2454] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:34.711201 9285 raft_consensus.cc:2454] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:34.711035 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0779912d7a04b4c95dd1db6f9c4d6fd" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:34.711345 9669 raft_consensus.cc:3037] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.711524 7078 leader_election.cc:302] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:34.711660 10894 raft_consensus.cc:2781] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:34.711742 10894 raft_consensus.cc:686] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:34.711879 10894 consensus_queue.cc:237] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.712188 9669 raft_consensus.cc:2454] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:34.713663 6737 catalog_manager.cc:5484] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:34.740311 9917 raft_consensus.cc:482] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:34.740418 9917 raft_consensus.cc:504] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.740869 9917 leader_election.cc:288] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:34.741302 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd67a40e0b834a83b41259ee4da3f3db" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:34.741317 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd67a40e0b834a83b41259ee4da3f3db" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:34.741428 9669 raft_consensus.cc:2454] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:34.741446 7142 raft_consensus.cc:2454] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:34.741823 9482 leader_election.cc:302] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:34.741988 9917 raft_consensus.cc:2781] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:34.742039 9917 raft_consensus.cc:482] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:34.742079 9917 raft_consensus.cc:3037] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.743157 9917 raft_consensus.cc:504] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.743451 9917 leader_election.cc:288] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:34.743810 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd67a40e0b834a83b41259ee4da3f3db" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:34.743944 7142 raft_consensus.cc:3037] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.744004 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd67a40e0b834a83b41259ee4da3f3db" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:34.744112 9669 raft_consensus.cc:3037] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.744673 7142 raft_consensus.cc:2454] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:34.744956 9482 leader_election.cc:302] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:34.744971 9669 raft_consensus.cc:2454] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:34.745340 9917 raft_consensus.cc:2781] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:34.745434 9917 raft_consensus.cc:686] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:34.745575 9917 consensus_queue.cc:237] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:34.747810 6737 catalog_manager.cc:5484] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:34.769158 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:34.773365 9285 raft_consensus.cc:1261] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:34.773378 9669 raft_consensus.cc:1261] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:34.773797 9917 consensus_queue.cc:1035] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.773941 10912 consensus_queue.cc:1035] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:34.776448 6734 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet 1c210fb4ffdf429ab95be6b01603ecc5 (table rebalancer_dynamic_table [id=93eb47e7e91041e0b31600c7f9f23551]). Expected version 1 got 0
I20240426 09:49:34.776829 11442 tablet.cc:1722] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:34.777029 6737 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet 1c210fb4ffdf429ab95be6b01603ecc5 (table rebalancer_dynamic_table [id=93eb47e7e91041e0b31600c7f9f23551]). Expected version 1 got 0
I20240426 09:49:34.777431 11442 tablet.cc:1620] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:34.777598 11443 tablet.cc:1722] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:34.777684 10960 tablet.cc:1722] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:34.777927 11443 tablet.cc:1620] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:34.777967 10960 tablet.cc:1620] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:34.783253 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=93eb47e7e91041e0b31600c7f9f23551] alter complete (version 1)
I20240426 09:49:34.887945 6734 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:34.893149 11442 tablet.cc:1722] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:34.893527 11442 tablet.cc:1620] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:34.893539 11443 tablet.cc:1722] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:34.893656 10960 tablet.cc:1722] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:34.893827 11443 tablet.cc:1620] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:34.893908 10960 tablet.cc:1620] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:34.898697 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=93eb47e7e91041e0b31600c7f9f23551] alter complete (version 2)
I20240426 09:49:34.928961 9917 raft_consensus.cc:482] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:34.929081 9917 raft_consensus.cc:504] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.929462 9917 leader_election.cc:288] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:34.929775 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb2e6c9a044463aa58878e06fc3fff2" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:34.929802 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb2e6c9a044463aa58878e06fc3fff2" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:34.929914 9285 raft_consensus.cc:2454] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:34.929939 9416 raft_consensus.cc:2454] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:34.930177 9482 leader_election.cc:302] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:34.930370 9917 raft_consensus.cc:2781] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:34.930413 9917 raft_consensus.cc:482] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:34.930436 9917 raft_consensus.cc:3037] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.931499 9917 raft_consensus.cc:504] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.931834 9917 leader_election.cc:288] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:34.932253 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb2e6c9a044463aa58878e06fc3fff2" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:34.932348 9416 raft_consensus.cc:3037] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.933194 9416 raft_consensus.cc:2454] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:34.933508 9482 leader_election.cc:302] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:34.933647 9917 raft_consensus.cc:2781] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:34.933738 9917 raft_consensus.cc:686] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:34.933868 9917 consensus_queue.cc:237] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:34.936015 6738 catalog_manager.cc:5484] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:34.937137 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cb2e6c9a044463aa58878e06fc3fff2" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:34.937372 9285 raft_consensus.cc:3037] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:34.938724 9285 raft_consensus.cc:2454] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:34.978349 6993 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:34.979852 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:34.980348 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:34.981698 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:34.983851 6862 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:34.994429 9528 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:35.002326 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:35.004964 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:35.012045 11442 tablet.cc:1722] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:35.014475 11443 tablet.cc:1722] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:35.018177 10960 tablet.cc:1722] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:35.021106 9661 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:35.024086 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=93eb47e7e91041e0b31600c7f9f23551] alter complete (version 3)
I20240426 09:49:35.063616 6880 consensus_queue.cc:237] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.064790 7013 raft_consensus.cc:1261] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:35.064793 9416 raft_consensus.cc:1261] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:35.065167 10725 consensus_queue.cc:1035] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:35.065349 11296 consensus_queue.cc:1035] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:35.067040 11481 raft_consensus.cc:2932] T 7e130e7733534aedb221d7772a2d29ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.067394 9416 raft_consensus.cc:2932] T 7e130e7733534aedb221d7772a2d29ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.067448 7013 raft_consensus.cc:2932] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.070055 6738 catalog_manager.cc:5484] T 7e130e7733534aedb221d7772a2d29ce P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.073734 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e130e7733534aedb221d7772a2d29ce with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:35.073951 11208 tablet_replica.cc:330] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:35.074090 11208 raft_consensus.cc:2229] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.074273 11208 raft_consensus.cc:2258] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.074740 11208 ts_tablet_manager.cc:1875] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:35.076879 11208 ts_tablet_manager.cc:1888] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:35.076961 11208 log.cc:1196] T 7e130e7733534aedb221d7772a2d29ce P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/7e130e7733534aedb221d7772a2d29ce
I20240426 09:49:35.077427 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 7e130e7733534aedb221d7772a2d29ce (table rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]) successfully deleted
I20240426 09:49:35.094652 6737 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240426 09:49:35.094849 6737 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240426 09:49:35.097416 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet c31c4fd362e84114b8dc45faa0ee8383
I20240426 09:49:35.097779 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 1c2fd1740d544f5d98741aecb64cdfc7
I20240426 09:49:35.098031 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet c8ab2d56585a450a8b2d7d9497f7c9cd
I20240426 09:49:35.098093 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet c31c4fd362e84114b8dc45faa0ee8383 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:35.098102 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet c31c4fd362e84114b8dc45faa0ee8383 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:35.098289 10519 tablet_replica.cc:330] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:35.098313 10689 tablet_replica.cc:330] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:35.098428 10689 raft_consensus.cc:2229] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:35.098433 10519 raft_consensus.cc:2229] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.098551 10519 raft_consensus.cc:2258] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.098606 10689 raft_consensus.cc:2258] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.098743 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2fd1740d544f5d98741aecb64cdfc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:35.098959 11208 tablet_replica.cc:330] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:35.099103 11208 raft_consensus.cc:2229] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.099135 10519 ts_tablet_manager.cc:1875] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.099135 10689 ts_tablet_manager.cc:1875] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.099263 11208 raft_consensus.cc:2258] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.101269 10689 ts_tablet_manager.cc:1888] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:35.101269 10519 ts_tablet_manager.cc:1888] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:35.101333 10689 log.cc:1196] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/c31c4fd362e84114b8dc45faa0ee8383
I20240426 09:49:35.101333 10519 log.cc:1196] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/c31c4fd362e84114b8dc45faa0ee8383
I20240426 09:49:35.101603 10519 ts_tablet_manager.cc:1909] T c31c4fd362e84114b8dc45faa0ee8383 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:35.101603 10689 ts_tablet_manager.cc:1909] T c31c4fd362e84114b8dc45faa0ee8383 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:35.102038 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2fd1740d544f5d98741aecb64cdfc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:35.102293 11498 tablet_replica.cc:330] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:35.102313 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ab2d56585a450a8b2d7d9497f7c9cd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:35.102325 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ab2d56585a450a8b2d7d9497f7c9cd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:35.102422 11498 raft_consensus.cc:2229] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.102501 11208 ts_tablet_manager.cc:1875] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.102530 11498 raft_consensus.cc:2258] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.102756 6860 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c2fd1740d544f5d98741aecb64cdfc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:35.102931 6861 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ab2d56585a450a8b2d7d9497f7c9cd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:35.102964 10519 tablet_replica.cc:330] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:35.103085 10519 raft_consensus.cc:2229] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.103139 11498 ts_tablet_manager.cc:1875] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.103210 10519 raft_consensus.cc:2258] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.103467 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet c31c4fd362e84114b8dc45faa0ee8383 (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:35.103720 10519 ts_tablet_manager.cc:1875] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.104470 11208 ts_tablet_manager.cc:1888] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240426 09:49:35.104538 11208 log.cc:1196] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/1c2fd1740d544f5d98741aecb64cdfc7
I20240426 09:49:35.104777 11208 ts_tablet_manager.cc:1909] T 1c2fd1740d544f5d98741aecb64cdfc7 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:35.105334 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet c31c4fd362e84114b8dc45faa0ee8383 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:35.105378 11498 ts_tablet_manager.cc:1888] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240426 09:49:35.105404 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 1c2fd1740d544f5d98741aecb64cdfc7 (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:35.105448 11498 log.cc:1196] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/1c2fd1740d544f5d98741aecb64cdfc7
I20240426 09:49:35.105489 11414 tablet_replica.cc:330] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:35.105595 11414 raft_consensus.cc:2229] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:35.105610 10519 ts_tablet_manager.cc:1888] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:35.105671 10519 log.cc:1196] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/c8ab2d56585a450a8b2d7d9497f7c9cd
I20240426 09:49:35.105715 11498 ts_tablet_manager.cc:1909] T 1c2fd1740d544f5d98741aecb64cdfc7 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:35.105753 11414 raft_consensus.cc:2258] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.105909 10519 ts_tablet_manager.cc:1909] T c8ab2d56585a450a8b2d7d9497f7c9cd P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:35.106405 10689 tablet_replica.cc:330] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:35.106405 11414 ts_tablet_manager.cc:1875] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.106535 10689 raft_consensus.cc:2229] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.106670 10689 raft_consensus.cc:2258] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.107139 10689 ts_tablet_manager.cc:1875] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.108618 11414 ts_tablet_manager.cc:1888] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240426 09:49:35.108695 11414 log.cc:1196] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/1c2fd1740d544f5d98741aecb64cdfc7
I20240426 09:49:35.108948 11414 ts_tablet_manager.cc:1909] T 1c2fd1740d544f5d98741aecb64cdfc7 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:35.108983 11378 consensus_queue.cc:1035] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:35.109438 10689 ts_tablet_manager.cc:1888] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:35.109501 10689 log.cc:1196] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/c8ab2d56585a450a8b2d7d9497f7c9cd
I20240426 09:49:35.109623 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 1c2fd1740d544f5d98741aecb64cdfc7 (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:35.109760 10689 ts_tablet_manager.cc:1909] T c8ab2d56585a450a8b2d7d9497f7c9cd P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:35.110080 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet c8ab2d56585a450a8b2d7d9497f7c9cd (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:35.110672 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet c31c4fd362e84114b8dc45faa0ee8383 (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:35.110818 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 1c2fd1740d544f5d98741aecb64cdfc7 (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:35.110949 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet c8ab2d56585a450a8b2d7d9497f7c9cd (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:35.111128 11414 tablet_replica.cc:330] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:35.111274 11414 raft_consensus.cc:2229] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:35.111466 11414 raft_consensus.cc:2258] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.111932 11414 ts_tablet_manager.cc:1875] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.113624 11414 ts_tablet_manager.cc:1888] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:35.113687 11414 log.cc:1196] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/c8ab2d56585a450a8b2d7d9497f7c9cd
I20240426 09:49:35.113914 11414 ts_tablet_manager.cc:1909] T c8ab2d56585a450a8b2d7d9497f7c9cd P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:35.114363 11414 tablet_replica.cc:330] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:35.114497 11414 raft_consensus.cc:2229] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.114503 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet c8ab2d56585a450a8b2d7d9497f7c9cd (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:35.114616 11414 raft_consensus.cc:2258] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.115047 11414 ts_tablet_manager.cc:1875] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.116479 11414 ts_tablet_manager.cc:1888] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240426 09:49:35.116554 11414 log.cc:1196] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/c31c4fd362e84114b8dc45faa0ee8383
I20240426 09:49:35.116813 11414 ts_tablet_manager.cc:1909] T c31c4fd362e84114b8dc45faa0ee8383 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:35.117429 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet c31c4fd362e84114b8dc45faa0ee8383 (table rebalance_test_table_18 [id=2e82674305404cf89d917b576af93457]) successfully deleted
I20240426 09:49:35.119436 10894 consensus_queue.cc:1035] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:35.129058 6738 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:35.129199 6738 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:35.131366 6738 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 1c210fb4ffdf429ab95be6b01603ecc5
I20240426 09:49:35.132100 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c210fb4ffdf429ab95be6b01603ecc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:35.132162 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c210fb4ffdf429ab95be6b01603ecc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:35.132266 11498 tablet_replica.cc:330] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:35.132232 9528 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c210fb4ffdf429ab95be6b01603ecc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:35.132391 11498 raft_consensus.cc:2229] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.132526 11498 raft_consensus.cc:2258] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.132558 11525 tablet_replica.cc:330] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:35.132575 11524 tablet_replica.cc:330] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:35.132678 11525 raft_consensus.cc:2229] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:35.132880 11525 raft_consensus.cc:2258] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.133828 11498 ts_tablet_manager.cc:1875] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.136058 11498 ts_tablet_manager.cc:1888] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:35.136600 11525 ts_tablet_manager.cc:1875] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.138201 11498 log.cc:1196] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/1c210fb4ffdf429ab95be6b01603ecc5
I20240426 09:49:35.138487 11498 ts_tablet_manager.cc:1909] T 1c210fb4ffdf429ab95be6b01603ecc5 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:35.139366 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 1c210fb4ffdf429ab95be6b01603ecc5 (table rebalancer_dynamic_table_new_name [id=93eb47e7e91041e0b31600c7f9f23551]) successfully deleted
I20240426 09:49:35.139847 11525 ts_tablet_manager.cc:1888] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:35.139978 11525 log.cc:1196] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/1c210fb4ffdf429ab95be6b01603ecc5
I20240426 09:49:35.140347 11525 ts_tablet_manager.cc:1909] T 1c210fb4ffdf429ab95be6b01603ecc5 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:35.140450 11524 raft_consensus.cc:2229] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.140738 11524 raft_consensus.cc:2258] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.141505 11524 ts_tablet_manager.cc:1875] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.141618 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 1c210fb4ffdf429ab95be6b01603ecc5 (table rebalancer_dynamic_table_new_name [id=93eb47e7e91041e0b31600c7f9f23551]) successfully deleted
I20240426 09:49:35.145399 11524 ts_tablet_manager.cc:1888] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:35.145529 11524 log.cc:1196] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/1c210fb4ffdf429ab95be6b01603ecc5
I20240426 09:49:35.145866 11524 ts_tablet_manager.cc:1909] T 1c210fb4ffdf429ab95be6b01603ecc5 P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:35.146561 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 1c210fb4ffdf429ab95be6b01603ecc5 (table rebalancer_dynamic_table_new_name [id=93eb47e7e91041e0b31600c7f9f23551]) successfully deleted
I20240426 09:49:35.167927 9396 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:35.168151 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:35.168226 6993 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:35.173789 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:35.178959 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:35.179216 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:35.180035 9661 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:35.182988 9528 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:35.233008 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:35.239483 10912 consensus_queue.cc:1035] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:35.239717 6993 tablet_service.cc:1467] Processing CreateTablet for tablet cfcbaf5eeb244932a356ecde47771ffc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=fcf8329d3a8b45649f09cd74e3b7eb32]), partition=
I20240426 09:49:35.239992 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfcbaf5eeb244932a356ecde47771ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.242439 11548 tablet_bootstrap.cc:492] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:35.242869 10912 consensus_queue.cc:1035] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:35.243845 9528 tablet_service.cc:1467] Processing CreateTablet for tablet cfcbaf5eeb244932a356ecde47771ffc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=fcf8329d3a8b45649f09cd74e3b7eb32]), partition=
I20240426 09:49:35.243939 11548 tablet_bootstrap.cc:654] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.243963 9396 tablet_service.cc:1467] Processing CreateTablet for tablet cfcbaf5eeb244932a356ecde47771ffc (DEFAULT_TABLE table=rebalancer_dynamic_table [id=fcf8329d3a8b45649f09cd74e3b7eb32]), partition=
I20240426 09:49:35.244088 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfcbaf5eeb244932a356ecde47771ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.244204 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfcbaf5eeb244932a356ecde47771ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.245833 11550 tablet_bootstrap.cc:492] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:35.246841 11550 tablet_bootstrap.cc:654] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.247689 11551 tablet_bootstrap.cc:492] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:35.248642 11551 tablet_bootstrap.cc:654] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.249307 11548 tablet_bootstrap.cc:492] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:35.249403 11548 ts_tablet_manager.cc:1367] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:35.249872 11548 raft_consensus.cc:348] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.250021 11548 raft_consensus.cc:374] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.250067 11548 raft_consensus.cc:729] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.250202 11548 consensus_queue.cc:260] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.250475 11548 ts_tablet_manager.cc:1398] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:35.251015 11551 tablet_bootstrap.cc:492] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:35.251139 11551 ts_tablet_manager.cc:1367] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20240426 09:49:35.251708 11550 tablet_bootstrap.cc:492] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:35.251828 11550 ts_tablet_manager.cc:1367] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240426 09:49:35.252346 11550 raft_consensus.cc:348] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.252475 11550 raft_consensus.cc:374] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.252506 11550 raft_consensus.cc:729] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.252633 11550 consensus_queue.cc:260] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.252925 11550 ts_tablet_manager.cc:1398] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:35.253078 11551 raft_consensus.cc:348] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.253190 11551 raft_consensus.cc:374] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.253243 11551 raft_consensus.cc:729] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.253363 11551 consensus_queue.cc:260] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.253633 11551 ts_tablet_manager.cc:1398] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:35.258553 7013 consensus_queue.cc:237] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.259864 9416 raft_consensus.cc:1261] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:35.259920 7142 raft_consensus.cc:1261] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:35.260316 11561 consensus_queue.cc:1035] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:35.260489 11210 consensus_queue.cc:1035] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:35.262384 11561 raft_consensus.cc:2932] T 11a10ff62d224cf49fcb71d11b029537 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.262909 7142 raft_consensus.cc:2932] T 11a10ff62d224cf49fcb71d11b029537 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.263046 9416 raft_consensus.cc:2932] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.265867 6740 catalog_manager.cc:5484] T 11a10ff62d224cf49fcb71d11b029537 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.268729 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 11a10ff62d224cf49fcb71d11b029537 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:35.269155 11208 tablet_replica.cc:330] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:35.269300 11208 raft_consensus.cc:2229] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.269447 11208 raft_consensus.cc:2258] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.269909 11208 ts_tablet_manager.cc:1875] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:35.272199 11208 ts_tablet_manager.cc:1888] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:35.272271 11208 log.cc:1196] T 11a10ff62d224cf49fcb71d11b029537 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/11a10ff62d224cf49fcb71d11b029537
I20240426 09:49:35.272789 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 11a10ff62d224cf49fcb71d11b029537 (table rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]) successfully deleted
I20240426 09:49:35.314541 10912 consensus_queue.cc:1035] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:35.317994 10912 consensus_queue.cc:1035] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:35.442248 10817 raft_consensus.cc:482] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:35.442402 10817 raft_consensus.cc:504] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.443367 10817 leader_election.cc:288] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:35.443776 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcbaf5eeb244932a356ecde47771ffc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:35.443918 7013 raft_consensus.cc:2454] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:35.444290 9350 leader_election.cc:302] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:35.444495 10817 raft_consensus.cc:2781] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:35.444554 10817 raft_consensus.cc:482] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:35.444586 10817 raft_consensus.cc:3037] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.445472 10817 raft_consensus.cc:504] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.445927 10817 leader_election.cc:288] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:35.446672 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcbaf5eeb244932a356ecde47771ffc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:35.446808 7013 raft_consensus.cc:3037] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.446785 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:35.447551 7013 raft_consensus.cc:2454] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:35.447829 9350 leader_election.cc:302] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:35.447999 10817 raft_consensus.cc:2781] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:35.448213 10817 raft_consensus.cc:686] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:35.448493 10817 consensus_queue.cc:237] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.450321 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcbaf5eeb244932a356ecde47771ffc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:35.450450 9548 raft_consensus.cc:3037] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.451511 9548 raft_consensus.cc:2454] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:35.451730 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcbaf5eeb244932a356ecde47771ffc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:35.451825 9549 raft_consensus.cc:2362] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1. Re-sending same reply.
I20240426 09:49:35.452818 6737 catalog_manager.cc:5484] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:35.453724 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 988b6b1007634a8980398e4f68dd4c4d (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:35.453951 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 753f199b5cb2426ba90e8ad003efcb32 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:35.454041 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 988b6b1007634a8980398e4f68dd4c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.454209 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753f199b5cb2426ba90e8ad003efcb32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.454203 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 279519370ee640fd9bbd3028008f1d5d (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:35.454402 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279519370ee640fd9bbd3028008f1d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.454150 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 988b6b1007634a8980398e4f68dd4c4d (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:35.454635 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 988b6b1007634a8980398e4f68dd4c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.456465 11610 tablet_bootstrap.cc:492] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:35.456579 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 753f199b5cb2426ba90e8ad003efcb32 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:35.456831 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753f199b5cb2426ba90e8ad003efcb32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.456885 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 279519370ee640fd9bbd3028008f1d5d (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:35.457085 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279519370ee640fd9bbd3028008f1d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.457430 11610 tablet_bootstrap.cc:654] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.458642 11610 tablet_bootstrap.cc:492] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:35.458711 11610 ts_tablet_manager.cc:1367] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:35.458899 11548 tablet_bootstrap.cc:492] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:35.459168 11610 raft_consensus.cc:348] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.459318 9528 tablet_service.cc:1467] Processing CreateTablet for tablet 753f199b5cb2426ba90e8ad003efcb32 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:35.459424 11610 raft_consensus.cc:374] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.459463 11610 raft_consensus.cc:729] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.459581 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753f199b5cb2426ba90e8ad003efcb32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.459574 11610 consensus_queue.cc:260] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.459817 11548 tablet_bootstrap.cc:654] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.459940 11610 ts_tablet_manager.cc:1398] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:35.460096 11610 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:35.461088 11610 tablet_bootstrap.cc:654] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.461400 11548 tablet_bootstrap.cc:492] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:35.461485 11548 ts_tablet_manager.cc:1367] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:35.461867 11548 raft_consensus.cc:348] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.461951 11548 raft_consensus.cc:374] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.461969 11548 raft_consensus.cc:729] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.462015 11550 tablet_bootstrap.cc:492] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:35.462069 11548 consensus_queue.cc:260] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.462858 11550 tablet_bootstrap.cc:654] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.463068 11548 ts_tablet_manager.cc:1398] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:35.463738 11610 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:35.463817 11610 ts_tablet_manager.cc:1367] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:35.464016 11550 tablet_bootstrap.cc:492] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:35.464107 11550 ts_tablet_manager.cc:1367] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:35.464221 11610 raft_consensus.cc:348] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.464316 11610 raft_consensus.cc:374] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.464370 11610 raft_consensus.cc:729] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.464493 11610 consensus_queue.cc:260] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.464522 11550 raft_consensus.cc:348] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.464632 11550 raft_consensus.cc:374] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.464664 11550 raft_consensus.cc:729] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.464802 11550 consensus_queue.cc:260] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.464848 11610 ts_tablet_manager.cc:1398] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:35.465095 11550 ts_tablet_manager.cc:1398] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:35.465667 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 988b6b1007634a8980398e4f68dd4c4d (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:35.465809 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 279519370ee640fd9bbd3028008f1d5d (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:35.465957 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 988b6b1007634a8980398e4f68dd4c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.466404 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279519370ee640fd9bbd3028008f1d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:35.468482 11551 tablet_bootstrap.cc:492] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:35.469400 11551 tablet_bootstrap.cc:654] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.469406 11610 tablet_bootstrap.cc:492] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:35.470407 11551 tablet_bootstrap.cc:492] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:35.470484 11610 tablet_bootstrap.cc:654] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.470486 11551 ts_tablet_manager.cc:1367] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:35.470942 11551 raft_consensus.cc:348] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.471050 11551 raft_consensus.cc:374] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.471091 11551 raft_consensus.cc:729] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.471203 11551 consensus_queue.cc:260] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.471486 11551 ts_tablet_manager.cc:1398] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:35.471652 11551 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:35.471940 11610 tablet_bootstrap.cc:492] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:35.472019 11610 ts_tablet_manager.cc:1367] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:35.472151 11609 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:35.472608 11610 raft_consensus.cc:348] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.472697 11610 raft_consensus.cc:374] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.472724 11610 raft_consensus.cc:729] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.472852 11610 consensus_queue.cc:260] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.473192 11610 ts_tablet_manager.cc:1398] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:35.475735 11609 tablet_bootstrap.cc:654] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.477083 11609 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:35.477154 11609 ts_tablet_manager.cc:1367] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:35.477552 11609 raft_consensus.cc:348] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.477632 11609 raft_consensus.cc:374] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.477651 11609 raft_consensus.cc:729] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.477752 11609 consensus_queue.cc:260] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.478015 11609 ts_tablet_manager.cc:1398] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:35.478161 11609 tablet_bootstrap.cc:492] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:35.478924 11609 tablet_bootstrap.cc:654] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.483270 11551 tablet_bootstrap.cc:654] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:35.485297 11551 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:35.485381 11551 ts_tablet_manager.cc:1367] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20240426 09:49:35.485540 11609 tablet_bootstrap.cc:492] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:35.485617 11609 ts_tablet_manager.cc:1367] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:35.485968 11551 raft_consensus.cc:348] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.486008 11609 raft_consensus.cc:348] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.486080 11551 raft_consensus.cc:374] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.486101 11609 raft_consensus.cc:374] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:35.486119 11551 raft_consensus.cc:729] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.486127 11609 raft_consensus.cc:729] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:35.486231 11609 consensus_queue.cc:260] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.486239 11551 consensus_queue.cc:260] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.486500 11551 ts_tablet_manager.cc:1398] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:35.486534 11609 ts_tablet_manager.cc:1398] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:35.490130 6993 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:35.490321 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:35.496237 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:35.501158 9266 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:35.503043 9661 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:35.505057 9397 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:35.510917 11358 raft_consensus.cc:482] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:35.511030 11358 raft_consensus.cc:504] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.511518 11358 leader_election.cc:288] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:35.511997 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "988b6b1007634a8980398e4f68dd4c4d" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:35.512128 7274 raft_consensus.cc:2454] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:35.512495 9221 leader_election.cc:302] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:35.512976 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:35.513180 11358 raft_consensus.cc:2781] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:35.513259 11358 raft_consensus.cc:482] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:35.513299 11358 raft_consensus.cc:3037] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.513710 9528 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:35.514413 11358 raft_consensus.cc:504] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.514782 11358 leader_election.cc:288] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:35.515796 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "988b6b1007634a8980398e4f68dd4c4d" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:35.515919 9415 raft_consensus.cc:3037] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.516887 9415 raft_consensus.cc:2454] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:35.516939 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "988b6b1007634a8980398e4f68dd4c4d" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:35.517091 7274 raft_consensus.cc:3037] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.518321 9219 leader_election.cc:302] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:35.518497 11358 raft_consensus.cc:2781] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:35.518638 11358 raft_consensus.cc:686] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:35.518828 11358 consensus_queue.cc:237] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.519675 7274 raft_consensus.cc:2454] T 988b6b1007634a8980398e4f68dd4c4d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:35.520018 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "988b6b1007634a8980398e4f68dd4c4d" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:35.520146 9416 raft_consensus.cc:2362] T 988b6b1007634a8980398e4f68dd4c4d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1. Re-sending same reply.
I20240426 09:49:35.524446 6735 catalog_manager.cc:5484] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.579301 11631 raft_consensus.cc:482] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:35.579416 11631 raft_consensus.cc:504] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.579854 11631 leader_election.cc:288] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:35.580103 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279519370ee640fd9bbd3028008f1d5d" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:35.580157 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279519370ee640fd9bbd3028008f1d5d" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:35.580258 7274 raft_consensus.cc:2454] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:35.580278 9416 raft_consensus.cc:2454] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:35.580549 9219 leader_election.cc:302] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:35.580724 11631 raft_consensus.cc:2781] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:35.580791 11631 raft_consensus.cc:482] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:35.580832 11631 raft_consensus.cc:3037] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.581760 11631 raft_consensus.cc:504] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.582118 11631 leader_election.cc:288] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:35.582412 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279519370ee640fd9bbd3028008f1d5d" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:35.582530 7274 raft_consensus.cc:3037] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.582607 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:35.583519 7274 raft_consensus.cc:2454] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:35.583729 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279519370ee640fd9bbd3028008f1d5d" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:35.583859 9416 raft_consensus.cc:3037] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.584753 9416 raft_consensus.cc:2454] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:35.585124 9221 leader_election.cc:302] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:35.585281 11631 raft_consensus.cc:2781] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:35.585359 11631 raft_consensus.cc:686] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:35.585484 11631 consensus_queue.cc:237] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:35.588093 6735 catalog_manager.cc:5484] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:35.588172 9549 raft_consensus.cc:1261] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:35.588640 7013 raft_consensus.cc:1261] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:35.588594 10817 consensus_queue.cc:1035] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:35.589147 10817 consensus_queue.cc:1035] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:35.591279 6732 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet cfcbaf5eeb244932a356ecde47771ffc (table rebalancer_dynamic_table [id=fcf8329d3a8b45649f09cd74e3b7eb32]). Expected version 1 got 0
I20240426 09:49:35.593338 6732 catalog_manager.cc:5576] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135) does not have the latest schema for tablet cfcbaf5eeb244932a356ecde47771ffc (table rebalancer_dynamic_table [id=fcf8329d3a8b45649f09cd74e3b7eb32]). Expected version 1 got 0
I20240426 09:49:35.594043 11340 mvcc.cc:205] Tried to move back new op lower bound from 7021055900000858112 to 7021055899439656960. Current Snapshot: MvccSnapshot[applied={T|T < 7021055900000858112}]
I20240426 09:49:35.594677 11652 tablet.cc:1722] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:35.595494 11652 tablet.cc:1620] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:35.595788 7274 consensus_queue.cc:237] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:35.595824 11653 tablet.cc:1722] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:35.597020 11653 tablet.cc:1620] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:35.597332 7142 raft_consensus.cc:1261] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:35.597555 6735 catalog_manager.cc:5576] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135) does not have the latest schema for tablet cfcbaf5eeb244932a356ecde47771ffc (table rebalancer_dynamic_table [id=fcf8329d3a8b45649f09cd74e3b7eb32]). Expected version 1 got 0
I20240426 09:49:35.597839 9285 raft_consensus.cc:1261] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:35.598475 10802 consensus_queue.cc:1035] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:35.598845 10802 consensus_queue.cc:1035] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:35.599365 11654 tablet.cc:1722] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:35.599642 11654 tablet.cc:1620] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:35.601258 10802 raft_consensus.cc:2932] T 295e6e95d86c47c88a09e47448c93d2e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:35.601732 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=fcf8329d3a8b45649f09cd74e3b7eb32] alter complete (version 1)
I20240426 09:49:35.603268 7142 raft_consensus.cc:2932] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:35.603349 9285 raft_consensus.cc:2932] T 295e6e95d86c47c88a09e47448c93d2e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:35.606570 6735 catalog_manager.cc:5484] T 295e6e95d86c47c88a09e47448c93d2e P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:35.607609 6732 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet cfcbaf5eeb244932a356ecde47771ffc (table rebalancer_dynamic_table [id=fcf8329d3a8b45649f09cd74e3b7eb32]). Expected version 1 got 0
I20240426 09:49:35.609982 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 295e6e95d86c47c88a09e47448c93d2e with delete_type TABLET_DATA_TOMBSTONED (TS c244e02f3cf847c0a5b69e1059f1071c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:35.610251 11414 tablet_replica.cc:330] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:35.610409 11414 raft_consensus.cc:2229] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.610560 11414 raft_consensus.cc:2258] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.611090 11414 ts_tablet_manager.cc:1875] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:35.613150 11414 ts_tablet_manager.cc:1888] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:35.613242 11414 log.cc:1196] T 295e6e95d86c47c88a09e47448c93d2e P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/295e6e95d86c47c88a09e47448c93d2e
I20240426 09:49:35.613626 10912 raft_consensus.cc:482] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:35.613721 10912 raft_consensus.cc:504] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.613792 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 295e6e95d86c47c88a09e47448c93d2e (table rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]) successfully deleted
I20240426 09:49:35.614099 10912 leader_election.cc:288] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:35.614387 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753f199b5cb2426ba90e8ad003efcb32" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:35.614534 7013 raft_consensus.cc:2454] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:35.614514 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753f199b5cb2426ba90e8ad003efcb32" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:35.614617 9285 raft_consensus.cc:2454] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:35.614861 9482 leader_election.cc:302] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:35.615038 10912 raft_consensus.cc:2781] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:35.615085 10912 raft_consensus.cc:482] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:35.615130 10912 raft_consensus.cc:3037] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.615736 10912 raft_consensus.cc:504] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.615981 10912 leader_election.cc:288] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:35.616274 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753f199b5cb2426ba90e8ad003efcb32" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:35.616266 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753f199b5cb2426ba90e8ad003efcb32" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:35.616364 9285 raft_consensus.cc:3037] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.616374 7013 raft_consensus.cc:3037] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:35.617262 7013 raft_consensus.cc:2454] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:35.617264 9285 raft_consensus.cc:2454] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:35.617501 9483 leader_election.cc:302] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:35.617691 10912 raft_consensus.cc:2781] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:35.617779 10912 raft_consensus.cc:686] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:35.617913 10912 consensus_queue.cc:237] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:35.619624 6732 catalog_manager.cc:5484] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:35.688719 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:35.690802 9397 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:35.693498 9661 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:35.696470 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:35.701232 6993 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:35.704514 9528 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:35.708678 9266 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:35.710115 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:35.718267 11652 tablet.cc:1722] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:35.718597 11652 tablet.cc:1620] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:35.718626 11654 tablet.cc:1722] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:35.718966 11654 tablet.cc:1620] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:35.723870 11653 tablet.cc:1722] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:35.724151 11653 tablet.cc:1620] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:35.724994 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=fcf8329d3a8b45649f09cd74e3b7eb32] alter complete (version 2)
I20240426 09:49:35.727897 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:35.781750 7013 consensus_queue.cc:237] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:35.783064 9669 raft_consensus.cc:1261] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:35.783095 7274 raft_consensus.cc:1261] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:35.783453 11561 consensus_queue.cc:1035] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:35.783633 11564 consensus_queue.cc:1035] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:35.785356 11564 raft_consensus.cc:2932] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:35.785776 9669 raft_consensus.cc:2932] T bd077eafe82d4531a6850e6686a7e566 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:35.786082 7274 raft_consensus.cc:2932] T bd077eafe82d4531a6850e6686a7e566 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:35.789072 6733 catalog_manager.cc:5484] T bd077eafe82d4531a6850e6686a7e566 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:35.791965 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet bd077eafe82d4531a6850e6686a7e566 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:35.792605 11732 tablet_replica.cc:330] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:35.792760 11732 raft_consensus.cc:2229] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.792919 11732 raft_consensus.cc:2258] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.793394 11732 ts_tablet_manager.cc:1875] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:35.795867 11732 ts_tablet_manager.cc:1888] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:35.795951 11732 log.cc:1196] T bd077eafe82d4531a6850e6686a7e566 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/bd077eafe82d4531a6850e6686a7e566
I20240426 09:49:35.796535 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet bd077eafe82d4531a6850e6686a7e566 (table rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]) successfully deleted
I20240426 09:49:35.829991 6733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:35.835702 11652 tablet.cc:1722] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:35.836175 11654 tablet.cc:1722] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:35.836210 11653 tablet.cc:1722] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:35.840617 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=fcf8329d3a8b45649f09cd74e3b7eb32] alter complete (version 3)
I20240426 09:49:35.881242 6993 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:35.882738 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:35.883992 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:35.885968 9661 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:35.893640 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:35.899540 9397 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:35.899960 9266 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:35.911289 9528 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:35.945398 6737 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:35.945549 6737 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:35.947755 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet cfcbaf5eeb244932a356ecde47771ffc
I20240426 09:49:35.948616 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet cfcbaf5eeb244932a356ecde47771ffc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:35.948616 9528 tablet_service.cc:1514] Processing DeleteTablet for tablet cfcbaf5eeb244932a356ecde47771ffc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:35.948890 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet cfcbaf5eeb244932a356ecde47771ffc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:35 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:35.949033 11780 tablet_replica.cc:330] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:35.949187 11780 raft_consensus.cc:2229] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.949314 11780 raft_consensus.cc:2258] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.949527 11781 tablet_replica.cc:330] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:35.949644 11781 raft_consensus.cc:2229] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:35.949746 11781 raft_consensus.cc:2258] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.949997 11780 ts_tablet_manager.cc:1875] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.950206 11782 tablet_replica.cc:330] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:35.950278 11781 ts_tablet_manager.cc:1875] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.950393 11782 raft_consensus.cc:2229] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:35.950824 11782 raft_consensus.cc:2258] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:35.951428 11782 ts_tablet_manager.cc:1875] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:35.952675 11781 ts_tablet_manager.cc:1888] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:35.952802 11781 log.cc:1196] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/cfcbaf5eeb244932a356ecde47771ffc
I20240426 09:49:35.953179 11781 ts_tablet_manager.cc:1909] T cfcbaf5eeb244932a356ecde47771ffc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:35.953740 11780 ts_tablet_manager.cc:1888] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:35.953814 11780 log.cc:1196] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/cfcbaf5eeb244932a356ecde47771ffc
I20240426 09:49:35.953930 11782 ts_tablet_manager.cc:1888] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:35.954023 11782 log.cc:1196] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/cfcbaf5eeb244932a356ecde47771ffc
I20240426 09:49:35.954008 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet cfcbaf5eeb244932a356ecde47771ffc (table rebalancer_dynamic_table_new_name [id=fcf8329d3a8b45649f09cd74e3b7eb32]) successfully deleted
I20240426 09:49:35.954107 11780 ts_tablet_manager.cc:1909] T cfcbaf5eeb244932a356ecde47771ffc P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:35.954321 11782 ts_tablet_manager.cc:1909] T cfcbaf5eeb244932a356ecde47771ffc P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:35.954828 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet cfcbaf5eeb244932a356ecde47771ffc (table rebalancer_dynamic_table_new_name [id=fcf8329d3a8b45649f09cd74e3b7eb32]) successfully deleted
I20240426 09:49:35.954989 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet cfcbaf5eeb244932a356ecde47771ffc (table rebalancer_dynamic_table_new_name [id=fcf8329d3a8b45649f09cd74e3b7eb32]) successfully deleted
I20240426 09:49:35.993649 7274 consensus_queue.cc:237] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
W20240426 09:49:35.995509 7210 consensus_peers.cc:485] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 722306427b7b46b4b0942193c821d990. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:35.995534 9669 raft_consensus.cc:1261] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:35.995566 9549 raft_consensus.cc:1261] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:35.996265 11656 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:35.996541 10802 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:35.998845 11656 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:35.999270 9669 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:36.000914 9549 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:36.001730 6737 catalog_manager.cc:5484] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:36.021356 6880 consensus_queue.cc:237] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:36.022958 7013 raft_consensus.cc:1261] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:36.023054 7142 raft_consensus.cc:1261] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:36.023103 6815 consensus_peers.cc:485] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0a674c2acbe4f69a7b8fd30fa12b833. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:36.023360 11818 consensus_queue.cc:1035] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:36.023562 11481 consensus_queue.cc:1035] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:36.025357 11818 raft_consensus.cc:2932] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:36.025964 7142 raft_consensus.cc:2932] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:36.026623 11358 consensus_queue.cc:1035] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.029491 6733 catalog_manager.cc:5484] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:36.029820 7013 raft_consensus.cc:2932] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:36.034476 11631 consensus_queue.cc:1035] T 988b6b1007634a8980398e4f68dd4c4d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.049775 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:36.051218 9285 consensus_queue.cc:237] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:36.053205 9669 raft_consensus.cc:1261] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:36.053242 9416 raft_consensus.cc:1261] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:36.053434 9219 consensus_peers.cc:485] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b5e68d9b3ce49d3aa398e85e139ddbf. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:36.053660 11631 consensus_queue.cc:1035] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:36.053822 11842 consensus_queue.cc:1035] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:36.055905 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 908c23fd41a54ea2aeedc84b9f78ae1c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=616bfccec202434998501504623d50eb]), partition=
I20240426 09:49:36.056022 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 908c23fd41a54ea2aeedc84b9f78ae1c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=616bfccec202434998501504623d50eb]), partition=
I20240426 09:49:36.056182 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908c23fd41a54ea2aeedc84b9f78ae1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.056262 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908c23fd41a54ea2aeedc84b9f78ae1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.056246 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 908c23fd41a54ea2aeedc84b9f78ae1c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=616bfccec202434998501504623d50eb]), partition=
I20240426 09:49:36.056459 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908c23fd41a54ea2aeedc84b9f78ae1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.058084 11845 tablet_bootstrap.cc:492] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:36.059142 11847 tablet_bootstrap.cc:492] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:36.059230 11631 raft_consensus.cc:2932] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:36.060336 11845 tablet_bootstrap.cc:654] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.060842 11847 tablet_bootstrap.cc:654] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.061389 11845 tablet_bootstrap.cc:492] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:36.061627 11845 ts_tablet_manager.cc:1367] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:36.061878 9416 raft_consensus.cc:2932] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:36.062044 11847 tablet_bootstrap.cc:492] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:36.062117 11847 ts_tablet_manager.cc:1367] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:36.062281 11845 raft_consensus.cc:348] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.062371 11845 raft_consensus.cc:374] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.062309 6737 catalog_manager.cc:5484] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:36.062443 11845 raft_consensus.cc:729] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.062579 11845 consensus_queue.cc:260] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.062595 11847 raft_consensus.cc:348] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.062700 11847 raft_consensus.cc:374] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.062734 11847 raft_consensus.cc:729] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.062961 11845 ts_tablet_manager.cc:1398] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.062906 11847 consensus_queue.cc:260] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.063489 11847 ts_tablet_manager.cc:1398] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.066363 9669 raft_consensus.cc:2932] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:36.068327 11846 tablet_bootstrap.cc:492] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:36.070297 11846 tablet_bootstrap.cc:654] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.072417 11846 tablet_bootstrap.cc:492] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:36.072575 11846 ts_tablet_manager.cc:1367] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240426 09:49:36.073138 11846 raft_consensus.cc:348] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.073351 11846 raft_consensus.cc:374] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.073410 11846 raft_consensus.cc:729] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.073613 11846 consensus_queue.cc:260] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.073917 11846 ts_tablet_manager.cc:1398] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:36.087045 11848 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.087657 9669 consensus_queue.cc:237] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.089437 9285 raft_consensus.cc:1261] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:36.089828 11869 consensus_queue.cc:1035] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:36.089968 6880 raft_consensus.cc:1261] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:36.090353 10201 consensus_queue.cc:1035] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:36.090552 11358 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240426 09:49:36.092231 9616 consensus_peers.cc:485] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce43c4c9b2994813ad7ad38f2e1b1040. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:36.092339 11869 raft_consensus.cc:2932] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:36.092643 6880 raft_consensus.cc:2932] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:36.095302 6737 catalog_manager.cc:5484] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:36.095324 9285 raft_consensus.cc:2932] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:36.131839 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:36.138638 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 2770cd77ebb642e3a52183a3871e33fa (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.138689 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 2770cd77ebb642e3a52183a3871e33fa (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.139014 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2770cd77ebb642e3a52183a3871e33fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.139058 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2770cd77ebb642e3a52183a3871e33fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.140152 9660 tablet_service.cc:1467] Processing CreateTablet for tablet ca0a2b121aba4659bf2d3a475a1e298b (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.140631 9528 tablet_service.cc:1467] Processing CreateTablet for tablet fb3c46383718465783af89bb0ad4896c (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.140913 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3c46383718465783af89bb0ad4896c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.141589 11846 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:36.142666 11846 tablet_bootstrap.cc:654] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.144011 6861 tablet_service.cc:1467] Processing CreateTablet for tablet ca0a2b121aba4659bf2d3a475a1e298b (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.144317 6861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0a2b121aba4659bf2d3a475a1e298b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.145159 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0a2b121aba4659bf2d3a475a1e298b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.146345 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 2770cd77ebb642e3a52183a3871e33fa (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.146584 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2770cd77ebb642e3a52183a3871e33fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.146553 6860 tablet_service.cc:1467] Processing CreateTablet for tablet fb3c46383718465783af89bb0ad4896c (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.146797 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3c46383718465783af89bb0ad4896c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.147063 7255 tablet_service.cc:1467] Processing CreateTablet for tablet ca0a2b121aba4659bf2d3a475a1e298b (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.147344 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0a2b121aba4659bf2d3a475a1e298b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.149269 9397 tablet_service.cc:1467] Processing CreateTablet for tablet fb3c46383718465783af89bb0ad4896c (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.146368 11847 tablet_bootstrap.cc:492] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:36.149600 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3c46383718465783af89bb0ad4896c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.150332 11847 tablet_bootstrap.cc:654] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.152545 11846 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:36.152631 11846 ts_tablet_manager.cc:1367] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20240426 09:49:36.153117 11846 raft_consensus.cc:348] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:36.153247 11846 raft_consensus.cc:374] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.153270 11846 raft_consensus.cc:729] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.153376 11846 consensus_queue.cc:260] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:36.153657 11846 ts_tablet_manager.cc:1398] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:36.153934 11916 tablet_bootstrap.cc:492] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:36.154192 11846 tablet_bootstrap.cc:492] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:36.154618 11914 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:36.156883 11847 tablet_bootstrap.cc:492] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:36.157001 11847 ts_tablet_manager.cc:1367] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:36.157460 11847 raft_consensus.cc:348] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.157573 11847 raft_consensus.cc:374] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.157610 11847 raft_consensus.cc:729] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.157723 11847 consensus_queue.cc:260] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.158011 11847 ts_tablet_manager.cc:1398] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.158701 11847 tablet_bootstrap.cc:492] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:36.159556 11915 tablet_bootstrap.cc:492] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:36.159785 11916 tablet_bootstrap.cc:654] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.160295 11914 tablet_bootstrap.cc:654] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.160331 11847 tablet_bootstrap.cc:654] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.162039 11846 tablet_bootstrap.cc:654] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.161423 11915 tablet_bootstrap.cc:654] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.164515 11916 tablet_bootstrap.cc:492] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:36.164659 11916 ts_tablet_manager.cc:1367] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:36.165040 11846 tablet_bootstrap.cc:492] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:36.165134 11846 ts_tablet_manager.cc:1367] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20240426 09:49:36.165172 11916 raft_consensus.cc:348] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.165303 11916 raft_consensus.cc:374] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.165378 11916 raft_consensus.cc:729] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.165544 11916 consensus_queue.cc:260] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.165627 11846 raft_consensus.cc:348] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.165858 11916 ts_tablet_manager.cc:1398] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.165727 11846 raft_consensus.cc:374] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.166628 11914 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:36.166782 11914 ts_tablet_manager.cc:1367] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:49:36.167299 11914 raft_consensus.cc:348] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:36.167400 11846 raft_consensus.cc:729] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.167486 11914 raft_consensus.cc:374] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.167564 11914 raft_consensus.cc:729] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.167748 11914 consensus_queue.cc:260] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:36.168196 11914 ts_tablet_manager.cc:1398] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.168150 11846 consensus_queue.cc:260] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.168566 11914 tablet_bootstrap.cc:492] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:36.169428 11914 tablet_bootstrap.cc:654] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.170630 11915 tablet_bootstrap.cc:492] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:36.170895 11846 ts_tablet_manager.cc:1398] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20240426 09:49:36.170944 11915 ts_tablet_manager.cc:1367] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:36.171661 11915 raft_consensus.cc:348] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.171838 11915 raft_consensus.cc:374] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.171952 11915 raft_consensus.cc:729] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.172168 11915 consensus_queue.cc:260] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.172681 11915 ts_tablet_manager.cc:1398] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:36.173611 11914 tablet_bootstrap.cc:492] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:36.173743 11914 ts_tablet_manager.cc:1367] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:36.174364 11914 raft_consensus.cc:348] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.174542 11914 raft_consensus.cc:374] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.174615 11914 raft_consensus.cc:729] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.174834 11914 consensus_queue.cc:260] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.175256 11914 ts_tablet_manager.cc:1398] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.176224 11847 tablet_bootstrap.cc:492] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:36.176323 11847 ts_tablet_manager.cc:1367] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240426 09:49:36.176803 11847 raft_consensus.cc:348] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.176910 11847 raft_consensus.cc:374] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.176960 11847 raft_consensus.cc:729] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.177076 11847 consensus_queue.cc:260] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.177356 11847 ts_tablet_manager.cc:1398] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.177523 11847 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:36.178581 11847 tablet_bootstrap.cc:654] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.180141 11656 raft_consensus.cc:482] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:36.180249 11656 raft_consensus.cc:504] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.180413 11847 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:36.180481 11847 ts_tablet_manager.cc:1367] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:36.180639 11656 leader_election.cc:288] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:36.180939 11847 raft_consensus.cc:348] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:36.181031 11847 raft_consensus.cc:374] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.181087 11847 raft_consensus.cc:729] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.181073 11873 raft_consensus.cc:482] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:36.181146 11873 raft_consensus.cc:504] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.181216 11847 consensus_queue.cc:260] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:36.181460 11847 ts_tablet_manager.cc:1398] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.181496 11873 leader_election.cc:288] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:36.182224 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a2b121aba4659bf2d3a475a1e298b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:36.182322 6880 raft_consensus.cc:2454] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:36.182945 11818 raft_consensus.cc:482] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:36.183015 11818 raft_consensus.cc:504] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.183296 11818 leader_election.cc:288] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:36.183676 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3c46383718465783af89bb0ad4896c" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:36.183768 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3c46383718465783af89bb0ad4896c" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:36.183821 9548 raft_consensus.cc:2454] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:36.183908 9416 raft_consensus.cc:2454] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:36.184108 6815 leader_election.cc:302] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:36.184263 11818 raft_consensus.cc:2781] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:36.184329 11818 raft_consensus.cc:482] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:36.184365 11818 raft_consensus.cc:3037] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.184432 7210 leader_election.cc:302] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:36.184967 10802 raft_consensus.cc:2781] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:36.185038 10802 raft_consensus.cc:482] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:36.185071 10802 raft_consensus.cc:3037] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.185242 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a2b121aba4659bf2d3a475a1e298b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:36.185371 11818 raft_consensus.cc:504] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.185653 11818 leader_election.cc:288] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:36.185925 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3c46383718465783af89bb0ad4896c" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:36.186039 9548 raft_consensus.cc:3037] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.186139 10802 raft_consensus.cc:504] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.186398 7275 raft_consensus.cc:2379] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 558c4ecd0ece460d9591aa7a3c799d19 in current term 1: Already voted for candidate e3394a4be9ea4def8aa39b333c82c5ff in this term.
I20240426 09:49:36.186460 10802 leader_election.cc:288] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:36.187032 9548 raft_consensus.cc:2454] T fb3c46383718465783af89bb0ad4896c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:36.187011 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a2b121aba4659bf2d3a475a1e298b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:36.187108 9669 raft_consensus.cc:2454] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:36.187330 6815 leader_election.cc:302] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:36.187494 11818 raft_consensus.cc:2781] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:36.187577 11818 raft_consensus.cc:686] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:36.187688 11818 consensus_queue.cc:237] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.188305 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a2b121aba4659bf2d3a475a1e298b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:36.188407 6880 raft_consensus.cc:2454] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:36.188699 9616 leader_election.cc:302] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:36.189145 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3c46383718465783af89bb0ad4896c" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:36.189257 9416 raft_consensus.cc:3037] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.190016 9416 raft_consensus.cc:2454] T fb3c46383718465783af89bb0ad4896c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:36.189561 6734 catalog_manager.cc:5484] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:36.191196 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a2b121aba4659bf2d3a475a1e298b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:36.191279 11873 raft_consensus.cc:2781] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:36.191332 9669 raft_consensus.cc:3037] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.191494 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a2b121aba4659bf2d3a475a1e298b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:36.191581 6880 raft_consensus.cc:3037] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.192236 9669 raft_consensus.cc:2454] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:36.192574 6880 raft_consensus.cc:2454] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:36.192625 11873 raft_consensus.cc:482] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:36.192677 11873 raft_consensus.cc:3037] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:36.193039 7210 leader_election.cc:302] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:36.193289 11656 raft_consensus.cc:2781] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:36.193377 11656 raft_consensus.cc:686] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:36.193514 11656 consensus_queue.cc:237] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.193670 11873 raft_consensus.cc:504] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.194121 11873 leader_election.cc:288] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 2 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:36.194576 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a2b121aba4659bf2d3a475a1e298b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:36.195156 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0a2b121aba4659bf2d3a475a1e298b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:36.195251 6880 raft_consensus.cc:3037] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:36.196065 6880 raft_consensus.cc:2454] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 2.
I20240426 09:49:36.196328 9616 leader_election.cc:302] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters: e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:36.196542 11873 raft_consensus.cc:2781] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:36.196625 11873 raft_consensus.cc:686] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:36.196768 11873 consensus_queue.cc:237] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.196492 6734 catalog_manager.cc:5484] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:36.202970 6738 catalog_manager.cc:5484] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 1 to 2, leader changed from e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 2 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:36.208508 11358 raft_consensus.cc:482] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:36.208626 11358 raft_consensus.cc:504] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.209101 11358 leader_election.cc:288] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:36.209453 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908c23fd41a54ea2aeedc84b9f78ae1c" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:36.209476 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908c23fd41a54ea2aeedc84b9f78ae1c" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:36.209599 6880 raft_consensus.cc:2454] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:36.209621 9416 raft_consensus.cc:2454] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:36.209892 9221 leader_election.cc:302] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:36.210067 11358 raft_consensus.cc:2781] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:36.210129 11358 raft_consensus.cc:482] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:36.210182 11358 raft_consensus.cc:3037] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.211246 11358 raft_consensus.cc:504] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.211639 11358 leader_election.cc:288] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:36.211969 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908c23fd41a54ea2aeedc84b9f78ae1c" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:36.212086 9416 raft_consensus.cc:3037] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.212148 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908c23fd41a54ea2aeedc84b9f78ae1c" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:36.212242 6880 raft_consensus.cc:3037] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.212997 9416 raft_consensus.cc:2454] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:36.213207 6880 raft_consensus.cc:2454] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:36.213305 9219 leader_election.cc:302] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:36.213552 11358 raft_consensus.cc:2781] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:36.213649 11358 raft_consensus.cc:686] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:36.213801 11358 consensus_queue.cc:237] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.215633 6735 catalog_manager.cc:5484] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:36.222959 11939 consensus_queue.cc:1035] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.223745 10912 consensus_queue.cc:1035] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.235143 6993 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:36.237825 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:36.238468 9397 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:36.244750 9660 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:36.245302 9266 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:36.251657 9528 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:36.254966 6862 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:36.271458 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:36.333590 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:36.337749 9416 raft_consensus.cc:1261] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:36.337880 6880 raft_consensus.cc:1261] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:36.338192 11842 consensus_queue.cc:1035] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.338354 11358 consensus_queue.cc:1035] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.341969 6735 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 908c23fd41a54ea2aeedc84b9f78ae1c (table rebalancer_dynamic_table [id=616bfccec202434998501504623d50eb]). Expected version 1 got 0
I20240426 09:49:36.343323 6734 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 908c23fd41a54ea2aeedc84b9f78ae1c (table rebalancer_dynamic_table [id=616bfccec202434998501504623d50eb]). Expected version 1 got 0
I20240426 09:49:36.347843 11992 tablet.cc:1722] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:36.348436 6735 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 908c23fd41a54ea2aeedc84b9f78ae1c (table rebalancer_dynamic_table [id=616bfccec202434998501504623d50eb]). Expected version 1 got 0
I20240426 09:49:36.348714 11992 tablet.cc:1620] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:36.349166 11993 tablet.cc:1722] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:36.349483 11993 tablet.cc:1620] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:36.352845 11991 tablet.cc:1722] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:36.353173 11991 tablet.cc:1620] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:36.354547 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=616bfccec202434998501504623d50eb] alter complete (version 1)
I20240426 09:49:36.424311 11873 raft_consensus.cc:482] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:36.424454 11873 raft_consensus.cc:504] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:36.424872 11873 leader_election.cc:288] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:36.425225 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:36.425218 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:36.425343 9417 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:36.425362 6880 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:36.425616 9616 leader_election.cc:302] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:36.425756 11873 raft_consensus.cc:2781] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:36.425805 11873 raft_consensus.cc:482] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:36.425844 11873 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.426697 11873 raft_consensus.cc:504] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:36.427090 11873 leader_election.cc:288] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:36.427378 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:36.427403 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:36.427486 9417 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.427495 6880 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.428231 6880 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:36.428292 9417 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:36.428563 9616 leader_election.cc:302] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:36.428709 11873 raft_consensus.cc:2781] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:36.428809 11873 raft_consensus.cc:686] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:36.428936 11873 consensus_queue.cc:237] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:36.430241 11996 ts_tablet_manager.cc:902] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:36.430425 6735 catalog_manager.cc:5484] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:36.430707 11996 tablet_copy_client.cc:308] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:36.431731 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet b0a674c2acbe4f69a7b8fd30fa12b833 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:55679)
I20240426 09:49:36.431828 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet b0a674c2acbe4f69a7b8fd30fa12b833 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:55679: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-b0a674c2acbe4f69a7b8fd30fa12b833
I20240426 09:49:36.432873 6902 tablet_copy_source_session.cc:205] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:36.433681 11996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0a674c2acbe4f69a7b8fd30fa12b833. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.435559 11996 tablet_copy_client.cc:783] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:36.435745 11996 tablet_copy_client.cc:647] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:36.436760 11996 tablet_copy_client.cc:523] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:36.437672 11996 tablet_bootstrap.cc:492] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:36.453166 9266 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:36.457938 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:36.459105 6993 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:36.461687 6862 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:36.466701 11991 tablet.cc:1722] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:36.467023 11991 tablet.cc:1620] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:36.467099 11992 tablet.cc:1722] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:36.467357 11992 tablet.cc:1620] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:36.469769 9397 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:36.470081 9528 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:36.473817 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=616bfccec202434998501504623d50eb] alter complete (version 2)
I20240426 09:49:36.475679 11993 tablet.cc:1722] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:36.475988 11993 tablet.cc:1620] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:36.480834 9660 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:36.481750 11996 tablet_bootstrap.cc:492] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:36.482096 11996 tablet_bootstrap.cc:492] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:36.482313 11996 ts_tablet_manager.cc:1367] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.045s user 0.025s sys 0.004s
I20240426 09:49:36.482823 11996 raft_consensus.cc:348] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:36.483217 11996 raft_consensus.cc:729] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.483392 11996 consensus_queue.cc:260] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:36.483800 11996 ts_tablet_manager.cc:1398] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.484526 12013 ts_tablet_manager.cc:902] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:36.485358 12013 tablet_copy_client.cc:308] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:36.485833 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-b0a674c2acbe4f69a7b8fd30fa12b833 received from {username='slave'} at 127.0.99.135:55679
I20240426 09:49:36.485909 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-b0a674c2acbe4f69a7b8fd30fa12b833 on tablet b0a674c2acbe4f69a7b8fd30fa12b833 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:36.486133 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 722306427b7b46b4b0942193c821d990 from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:51655)
I20240426 09:49:36.486223 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 722306427b7b46b4b0942193c821d990 from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:51655: session id = c244e02f3cf847c0a5b69e1059f1071c-722306427b7b46b4b0942193c821d990
I20240426 09:49:36.487154 7295 tablet_copy_source_session.cc:205] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:36.488322 12013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722306427b7b46b4b0942193c821d990. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.490712 12013 tablet_copy_client.cc:783] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:36.490970 12013 tablet_copy_client.cc:647] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:36.492436 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:36.493760 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:36.528614 12013 tablet_copy_client.cc:523] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:36.530530 12013 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:36.534881 12013 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:36.535254 12013 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:49:36.535445 12013 ts_tablet_manager.cc:1367] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.002s
I20240426 09:49:36.535892 12013 raft_consensus.cc:348] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.536106 12013 raft_consensus.cc:729] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.536273 12013 consensus_queue.cc:260] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.536536 12013 ts_tablet_manager.cc:1398] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:36.537055 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-722306427b7b46b4b0942193c821d990 received from {username='slave'} at 127.0.99.129:51655
I20240426 09:49:36.537145 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-722306427b7b46b4b0942193c821d990 on tablet 722306427b7b46b4b0942193c821d990 with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:49:36.577911 6733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:36.583675 11991 tablet.cc:1722] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:36.584419 11992 tablet.cc:1722] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:36.586221 11993 tablet.cc:1722] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:36.589782 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=616bfccec202434998501504623d50eb] alter complete (version 3)
I20240426 09:49:36.598969 12059 ts_tablet_manager.cc:902] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:36.599751 12059 tablet_copy_client.cc:308] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:36.600478 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet ce43c4c9b2994813ad7ad38f2e1b1040 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:32841)
I20240426 09:49:36.600618 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet ce43c4c9b2994813ad7ad38f2e1b1040 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:32841: session id = e3394a4be9ea4def8aa39b333c82c5ff-ce43c4c9b2994813ad7ad38f2e1b1040
I20240426 09:49:36.601651 9685 tablet_copy_source_session.cc:205] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:36.602917 12059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce43c4c9b2994813ad7ad38f2e1b1040. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.612811 12059 tablet_copy_client.cc:783] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:36.613018 12059 tablet_copy_client.cc:647] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:36.615217 12059 tablet_copy_client.cc:523] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:36.616708 12059 tablet_bootstrap.cc:492] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:36.622045 12059 tablet_bootstrap.cc:492] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:36.622327 12059 tablet_bootstrap.cc:492] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:49:36.622844 12059 ts_tablet_manager.cc:1367] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.001s
I20240426 09:49:36.623554 12059 raft_consensus.cc:348] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.623728 12059 raft_consensus.cc:729] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.624346 12059 consensus_queue.cc:260] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.625082 12059 ts_tablet_manager.cc:1398] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:36.625870 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-ce43c4c9b2994813ad7ad38f2e1b1040 received from {username='slave'} at 127.0.99.132:32841
I20240426 09:49:36.625967 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-ce43c4c9b2994813ad7ad38f2e1b1040 on tablet ce43c4c9b2994813ad7ad38f2e1b1040 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:36.635680 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:36.635978 6993 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:36.638597 9660 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:36.641937 7275 raft_consensus.cc:3032] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:36.642015 7275 raft_consensus.cc:729] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:36.642210 7275 consensus_queue.cc:260] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:36.642390 7275 raft_consensus.cc:3037] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:36.643687 7275 raft_consensus.cc:1261] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20240426 09:49:36.643746 7275 pending_rounds.cc:76] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff: Aborting all ops after (but not including) 0
I20240426 09:49:36.643774 7275 pending_rounds.cc:97] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20240426 09:49:36.643801 7275 raft_consensus.cc:2866] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240426 09:49:36.644294 9397 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240426 09:49:36.644680 11873 consensus_queue.cc:1035] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.649739 6880 raft_consensus.cc:1261] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20240426 09:49:36.650085 11869 consensus_queue.cc:1035] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.650880 6862 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:36.663744 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:36.669549 9266 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:36.669731 12080 ts_tablet_manager.cc:902] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:36.672804 9528 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:36.679600 12080 tablet_copy_client.cc:308] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:49:36.680330 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet 5b5e68d9b3ce49d3aa398e85e139ddbf from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:39123)
I20240426 09:49:36.680459 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet 5b5e68d9b3ce49d3aa398e85e139ddbf from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:39123: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-5b5e68d9b3ce49d3aa398e85e139ddbf
I20240426 09:49:36.681746 9306 tablet_copy_source_session.cc:205] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:36.683284 12080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b5e68d9b3ce49d3aa398e85e139ddbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.685628 12080 tablet_copy_client.cc:783] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:36.687848 12080 tablet_copy_client.cc:647] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:36.691020 12080 tablet_copy_client.cc:523] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:36.692061 6738 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:36.692183 6738 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:36.694355 6738 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 908c23fd41a54ea2aeedc84b9f78ae1c
I20240426 09:49:36.695142 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 908c23fd41a54ea2aeedc84b9f78ae1c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:36 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:36.695176 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 908c23fd41a54ea2aeedc84b9f78ae1c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:36 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:36.695415 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 908c23fd41a54ea2aeedc84b9f78ae1c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:36 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:36.695660 12087 tablet_replica.cc:330] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:36.695668 12080 tablet_bootstrap.cc:492] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:36.695776 12087 raft_consensus.cc:2229] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:36.696031 12087 raft_consensus.cc:2258] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:36.696642 12087 ts_tablet_manager.cc:1875] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:36.698908 12087 ts_tablet_manager.cc:1888] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:36.699319 12087 log.cc:1196] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/908c23fd41a54ea2aeedc84b9f78ae1c
I20240426 09:49:36.700039 12087 ts_tablet_manager.cc:1909] T 908c23fd41a54ea2aeedc84b9f78ae1c P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:36.700701 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 908c23fd41a54ea2aeedc84b9f78ae1c (table rebalancer_dynamic_table_new_name [id=616bfccec202434998501504623d50eb]) successfully deleted
I20240426 09:49:36.705056 12085 tablet_replica.cc:330] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:36.705271 12085 raft_consensus.cc:2229] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:36.705475 12085 raft_consensus.cc:2258] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:36.706097 12085 ts_tablet_manager.cc:1875] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:36.706315 12086 tablet_replica.cc:330] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:36.708719 12085 ts_tablet_manager.cc:1888] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:36.708978 12086 raft_consensus.cc:2229] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:36.709162 12086 raft_consensus.cc:2258] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:36.709787 12086 ts_tablet_manager.cc:1875] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:36.712038 12086 ts_tablet_manager.cc:1888] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:36.712196 12085 log.cc:1196] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/908c23fd41a54ea2aeedc84b9f78ae1c
I20240426 09:49:36.712595 12085 ts_tablet_manager.cc:1909] T 908c23fd41a54ea2aeedc84b9f78ae1c P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:36.713351 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 908c23fd41a54ea2aeedc84b9f78ae1c (table rebalancer_dynamic_table_new_name [id=616bfccec202434998501504623d50eb]) successfully deleted
I20240426 09:49:36.713802 11818 consensus_queue.cc:1035] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.714313 12086 log.cc:1196] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/908c23fd41a54ea2aeedc84b9f78ae1c
I20240426 09:49:36.715361 12086 ts_tablet_manager.cc:1909] T 908c23fd41a54ea2aeedc84b9f78ae1c P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:36.716367 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 908c23fd41a54ea2aeedc84b9f78ae1c (table rebalancer_dynamic_table_new_name [id=616bfccec202434998501504623d50eb]) successfully deleted
I20240426 09:49:36.717557 12080 tablet_bootstrap.cc:492] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:36.717759 12080 tablet_bootstrap.cc:492] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:49:36.717873 12080 ts_tablet_manager.cc:1367] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20240426 09:49:36.718237 12080 raft_consensus.cc:348] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:36.718317 12080 raft_consensus.cc:729] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.718428 12080 consensus_queue.cc:260] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:36.719631 12080 ts_tablet_manager.cc:1398] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:36.720213 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-5b5e68d9b3ce49d3aa398e85e139ddbf received from {username='slave'} at 127.0.99.130:39123
I20240426 09:49:36.720206 11818 consensus_queue.cc:1035] T fb3c46383718465783af89bb0ad4896c P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.720307 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-5b5e68d9b3ce49d3aa398e85e139ddbf on tablet 5b5e68d9b3ce49d3aa398e85e139ddbf with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:49:36.741263 7275 consensus_queue.cc:237] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.742913 7013 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:36.742913 9285 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:36.743322 12108 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:36.743858 11656 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:36.745602 11656 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:36.746379 7013 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:36.748777 9285 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:36.749545 6738 catalog_manager.cc:5484] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:36.752782 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d42c52b7a94e32ab4cdad78ce884ee with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:36.753113 12112 tablet_replica.cc:330] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:36.753293 12112 raft_consensus.cc:2229] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:36.753446 12112 raft_consensus.cc:2258] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:36.753999 12112 ts_tablet_manager.cc:1875] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:36.756739 12112 ts_tablet_manager.cc:1888] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:36.756834 12112 log.cc:1196] T b2d42c52b7a94e32ab4cdad78ce884ee P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/b2d42c52b7a94e32ab4cdad78ce884ee
I20240426 09:49:36.757361 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet b2d42c52b7a94e32ab4cdad78ce884ee (table rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]) successfully deleted
I20240426 09:49:36.796023 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:36.801338 7123 tablet_service.cc:1467] Processing CreateTablet for tablet ce3730359fcf45c8b1264ac4ecbc05bb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e311ea2d8193486890fd3ee5f41e088f]), partition=
I20240426 09:49:36.801385 9528 tablet_service.cc:1467] Processing CreateTablet for tablet ce3730359fcf45c8b1264ac4ecbc05bb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e311ea2d8193486890fd3ee5f41e088f]), partition=
I20240426 09:49:36.801614 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3730359fcf45c8b1264ac4ecbc05bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.801635 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3730359fcf45c8b1264ac4ecbc05bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.803115 9266 tablet_service.cc:1467] Processing CreateTablet for tablet ce3730359fcf45c8b1264ac4ecbc05bb (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e311ea2d8193486890fd3ee5f41e088f]), partition=
I20240426 09:49:36.803367 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce3730359fcf45c8b1264ac4ecbc05bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.803718 12159 tablet_bootstrap.cc:492] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:36.804170 12160 tablet_bootstrap.cc:492] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:36.804744 12159 tablet_bootstrap.cc:654] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.805124 12160 tablet_bootstrap.cc:654] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.805703 12162 tablet_bootstrap.cc:492] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:36.805833 12159 tablet_bootstrap.cc:492] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:36.805917 12159 ts_tablet_manager.cc:1367] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:36.806710 12162 tablet_bootstrap.cc:654] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.807761 12159 raft_consensus.cc:348] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:36.807855 12160 tablet_bootstrap.cc:492] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:36.807914 12162 tablet_bootstrap.cc:492] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:36.807952 12159 raft_consensus.cc:374] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.808054 12159 raft_consensus.cc:729] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.807931 12160 ts_tablet_manager.cc:1367] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:36.808064 12162 ts_tablet_manager.cc:1367] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:36.808308 12159 consensus_queue.cc:260] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:36.808563 12160 raft_consensus.cc:348] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:36.808715 12160 raft_consensus.cc:374] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.808797 12160 raft_consensus.cc:729] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.808971 12160 consensus_queue.cc:260] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:36.809463 12160 ts_tablet_manager.cc:1398] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.809716 12159 ts_tablet_manager.cc:1398] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:36.810475 12162 raft_consensus.cc:348] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:36.810901 12162 raft_consensus.cc:374] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.811003 12162 raft_consensus.cc:729] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.811148 12162 consensus_queue.cc:260] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:36.812003 12162 ts_tablet_manager.cc:1398] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:36.819797 11869 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.822710 11869 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:36.861677 9548 raft_consensus.cc:1203] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:36.887722 6880 raft_consensus.cc:1203] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:36.955248 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:36.961069 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 957533c35bae4299b35b77ba9a465eb3 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.961285 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 8c623acd7cc9480faec733d2fd109117 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.961413 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957533c35bae4299b35b77ba9a465eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.961546 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c623acd7cc9480faec733d2fd109117. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.961670 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 957533c35bae4299b35b77ba9a465eb3 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.961926 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957533c35bae4299b35b77ba9a465eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.963850 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 08fa514a62a84c0e8068bfe23d175773 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.964075 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08fa514a62a84c0e8068bfe23d175773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.964365 12196 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:36.964677 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 08fa514a62a84c0e8068bfe23d175773 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.964955 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08fa514a62a84c0e8068bfe23d175773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.965411 12196 tablet_bootstrap.cc:654] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.966473 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 957533c35bae4299b35b77ba9a465eb3 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.966729 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957533c35bae4299b35b77ba9a465eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.966821 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 8c623acd7cc9480faec733d2fd109117 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.967067 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c623acd7cc9480faec733d2fd109117. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.967242 12199 tablet_bootstrap.cc:492] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:36.967424 9528 tablet_service.cc:1467] Processing CreateTablet for tablet 8c623acd7cc9480faec733d2fd109117 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.967681 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c623acd7cc9480faec733d2fd109117. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.968209 12199 tablet_bootstrap.cc:654] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.968897 12162 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:36.969354 12196 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:36.969441 12196 ts_tablet_manager.cc:1367] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:36.969732 12159 tablet_bootstrap.cc:492] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:36.969878 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 08fa514a62a84c0e8068bfe23d175773 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:36.969962 12196 raft_consensus.cc:348] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.970074 12196 raft_consensus.cc:374] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.970104 12196 raft_consensus.cc:729] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.970113 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08fa514a62a84c0e8068bfe23d175773. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:36.970302 12162 tablet_bootstrap.cc:654] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.970271 12196 consensus_queue.cc:260] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.970712 12159 tablet_bootstrap.cc:654] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.971622 12196 ts_tablet_manager.cc:1398] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:36.971757 12197 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:36.971959 12196 tablet_bootstrap.cc:492] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:36.972821 12197 tablet_bootstrap.cc:654] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.972980 12199 tablet_bootstrap.cc:492] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:36.973096 12199 ts_tablet_manager.cc:1367] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:36.973232 12162 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:36.973345 12162 ts_tablet_manager.cc:1367] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:36.973728 12199 raft_consensus.cc:348] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.973835 12199 raft_consensus.cc:374] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.973867 12199 raft_consensus.cc:729] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.973863 12162 raft_consensus.cc:348] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.973992 12162 raft_consensus.cc:374] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.974026 12162 raft_consensus.cc:729] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.973999 12199 consensus_queue.cc:260] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.974169 12162 consensus_queue.cc:260] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.974308 12199 ts_tablet_manager.cc:1398] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:36.974468 12162 ts_tablet_manager.cc:1398] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:36.974644 12162 tablet_bootstrap.cc:492] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:36.974964 12159 tablet_bootstrap.cc:492] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:36.975045 12159 ts_tablet_manager.cc:1367] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:36.975529 12162 tablet_bootstrap.cc:654] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.975466 12159 raft_consensus.cc:348] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.975569 12159 raft_consensus.cc:374] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.975598 12159 raft_consensus.cc:729] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.975728 12159 consensus_queue.cc:260] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.976032 12159 ts_tablet_manager.cc:1398] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.976208 12159 tablet_bootstrap.cc:492] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:36.976847 12196 tablet_bootstrap.cc:654] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.977068 12162 tablet_bootstrap.cc:492] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:36.977180 12162 ts_tablet_manager.cc:1367] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:36.977195 12159 tablet_bootstrap.cc:654] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.977627 12162 raft_consensus.cc:348] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.977725 12162 raft_consensus.cc:374] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.977756 12162 raft_consensus.cc:729] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.977876 12162 consensus_queue.cc:260] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.978153 12162 ts_tablet_manager.cc:1398] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:36.979168 12159 tablet_bootstrap.cc:492] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:36.979256 12159 ts_tablet_manager.cc:1367] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:36.979295 10912 raft_consensus.cc:482] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:36.979373 10912 raft_consensus.cc:504] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:36.979665 12159 raft_consensus.cc:348] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.979735 10912 leader_election.cc:288] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:36.979760 12197 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:36.979771 12159 raft_consensus.cc:374] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.979804 12159 raft_consensus.cc:729] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.979848 12197 ts_tablet_manager.cc:1367] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:36.979900 12159 consensus_queue.cc:260] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.980165 12159 ts_tablet_manager.cc:1398] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.980211 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3730359fcf45c8b1264ac4ecbc05bb" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:36.980325 9285 raft_consensus.cc:2454] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:36.980293 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3730359fcf45c8b1264ac4ecbc05bb" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:36.980415 7142 raft_consensus.cc:2454] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:36.980468 12197 raft_consensus.cc:348] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.980593 12197 raft_consensus.cc:374] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.980613 9483 leader_election.cc:302] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:36.980648 12197 raft_consensus.cc:729] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.980829 11939 raft_consensus.cc:2781] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:36.980891 11939 raft_consensus.cc:482] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:36.980929 11939 raft_consensus.cc:3037] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.981663 12197 consensus_queue.cc:260] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.982228 11939 raft_consensus.cc:504] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:36.982657 11939 leader_election.cc:288] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:36.982765 12197 ts_tablet_manager.cc:1398] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:36.983017 12197 tablet_bootstrap.cc:492] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:36.983004 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3730359fcf45c8b1264ac4ecbc05bb" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:36.983100 9285 raft_consensus.cc:3037] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.983618 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce3730359fcf45c8b1264ac4ecbc05bb" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:36.983752 7142 raft_consensus.cc:3037] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.983992 12197 tablet_bootstrap.cc:654] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:36.984061 9285 raft_consensus.cc:2454] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:36.984634 7142 raft_consensus.cc:2454] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:36.985045 9482 leader_election.cc:302] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:36.985235 11939 raft_consensus.cc:2781] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:36.985313 11939 raft_consensus.cc:686] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:36.985451 12196 tablet_bootstrap.cc:492] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:36.985437 11939 consensus_queue.cc:237] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:36.985524 12196 ts_tablet_manager.cc:1367] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240426 09:49:36.985935 12196 raft_consensus.cc:348] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.986023 12196 raft_consensus.cc:374] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.986052 12196 raft_consensus.cc:729] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.986184 12196 consensus_queue.cc:260] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.986491 12196 ts_tablet_manager.cc:1398] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:36.988111 12197 tablet_bootstrap.cc:492] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:36.988200 12197 ts_tablet_manager.cc:1367] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240426 09:49:36.988627 12197 raft_consensus.cc:348] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.988701 12197 raft_consensus.cc:374] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:36.988721 12197 raft_consensus.cc:729] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:36.988840 12197 consensus_queue.cc:260] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:36.989099 12197 ts_tablet_manager.cc:1398] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:36.990100 6737 catalog_manager.cc:5484] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:36.993139 11767 raft_consensus.cc:482] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:36.993232 11767 raft_consensus.cc:504] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.993713 11767 leader_election.cc:288] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:36.994040 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623acd7cc9480faec733d2fd109117" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:36.994166 9548 raft_consensus.cc:2454] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:36.994457 9350 leader_election.cc:302] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:36.998203 11767 raft_consensus.cc:2781] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:36.998275 11767 raft_consensus.cc:482] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:36.998322 11767 raft_consensus.cc:3037] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:36.999379 11767 raft_consensus.cc:504] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:36.999795 11767 leader_election.cc:288] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:37.000095 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623acd7cc9480faec733d2fd109117" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:37.000226 9548 raft_consensus.cc:3037] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.001291 9548 raft_consensus.cc:2454] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:37.002023 9350 leader_election.cc:302] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:37.002264 11767 raft_consensus.cc:2781] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:37.002346 11767 raft_consensus.cc:686] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:37.002458 11767 consensus_queue.cc:237] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:37.003975 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623acd7cc9480faec733d2fd109117" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:37.004127 9285 raft_consensus.cc:2454] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:37.004317 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623acd7cc9480faec733d2fd109117" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:37.004416 9286 raft_consensus.cc:3037] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.005430 9286 raft_consensus.cc:2454] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:37.006526 6735 catalog_manager.cc:5484] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.012866 9265 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:37.015200 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:37.018584 6862 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:37.019903 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:37.024461 9396 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240426 09:49:37.024986 6993 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:37.034389 9661 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:37.035305 9529 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:37.080090 6880 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b0a674c2acbe4f69a7b8fd30fa12b833"
dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54102
I20240426 09:49:37.080300 6880 raft_consensus.cc:595] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:37.088945 11939 raft_consensus.cc:482] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:37.089059 11939 raft_consensus.cc:504] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:37.089527 11939 leader_election.cc:288] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:37.089920 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08fa514a62a84c0e8068bfe23d175773" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:37.090024 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08fa514a62a84c0e8068bfe23d175773" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:37.090065 6880 raft_consensus.cc:2454] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:37.090169 9669 raft_consensus.cc:2454] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:37.090382 9484 leader_election.cc:302] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:37.090572 11939 raft_consensus.cc:2781] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:37.090633 11939 raft_consensus.cc:482] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:37.090673 11939 raft_consensus.cc:3037] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.091549 11939 raft_consensus.cc:504] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:37.091869 11939 leader_election.cc:288] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:37.092197 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08fa514a62a84c0e8068bfe23d175773" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:37.092206 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08fa514a62a84c0e8068bfe23d175773" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:37.092314 9669 raft_consensus.cc:3037] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.092314 6880 raft_consensus.cc:3037] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.093288 9669 raft_consensus.cc:2454] T 08fa514a62a84c0e8068bfe23d175773 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:37.093288 6880 raft_consensus.cc:2454] T 08fa514a62a84c0e8068bfe23d175773 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:37.093621 9482 leader_election.cc:302] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:37.093915 11939 raft_consensus.cc:2781] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:37.094025 11939 raft_consensus.cc:686] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:37.094156 11939 consensus_queue.cc:237] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:37.096051 6735 catalog_manager.cc:5484] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:37.099710 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:37.100222 11768 raft_consensus.cc:979] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: : Instructing follower 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:49:37.100324 11818 raft_consensus.cc:1067] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Signalling peer 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:49:37.100965 7013 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b0a674c2acbe4f69a7b8fd30fa12b833"
dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
from {username='slave'} at 127.0.99.129:51417
I20240426 09:49:37.101117 7013 raft_consensus.cc:482] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:37.101172 7013 raft_consensus.cc:3037] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:37.102130 7013 raft_consensus.cc:504] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:37.102710 7013 leader_election.cc:288] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:37.103314 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a674c2acbe4f69a7b8fd30fa12b833" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:37.103433 9548 raft_consensus.cc:3037] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:37.103453 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a674c2acbe4f69a7b8fd30fa12b833" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:37.103566 7142 raft_consensus.cc:3037] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:37.103574 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a674c2acbe4f69a7b8fd30fa12b833" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:37.103678 6880 raft_consensus.cc:3032] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:37.103713 6880 raft_consensus.cc:729] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:37.103850 6880 consensus_queue.cc:260] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:37.104024 6880 raft_consensus.cc:3037] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:37.104382 9548 raft_consensus.cc:2454] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:49:37.104501 7142 raft_consensus.cc:2454] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:49:37.104962 6880 raft_consensus.cc:2454] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:49:37.105279 6948 leader_election.cc:302] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:37.105544 12076 raft_consensus.cc:2781] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:37.105623 12076 raft_consensus.cc:686] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:37.105719 9286 raft_consensus.cc:1261] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:37.105752 12076 consensus_queue.cc:237] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:37.106165 11939 consensus_queue.cc:1035] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.106911 7142 raft_consensus.cc:1261] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:37.107350 10912 consensus_queue.cc:1035] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.108273 6735 catalog_manager.cc:5484] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 1 to 2, leader changed from c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:37.109772 6737 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet ce3730359fcf45c8b1264ac4ecbc05bb (table rebalancer_dynamic_table [id=e311ea2d8193486890fd3ee5f41e088f]). Expected version 1 got 0
I20240426 09:49:37.111274 12244 tablet.cc:1722] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:37.112061 12244 tablet.cc:1620] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:37.112427 12245 tablet.cc:1722] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:37.112792 6735 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet ce3730359fcf45c8b1264ac4ecbc05bb (table rebalancer_dynamic_table [id=e311ea2d8193486890fd3ee5f41e088f]). Expected version 1 got 0
I20240426 09:49:37.114114 6735 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet ce3730359fcf45c8b1264ac4ecbc05bb (table rebalancer_dynamic_table [id=e311ea2d8193486890fd3ee5f41e088f]). Expected version 1 got 0
I20240426 09:49:37.114856 12245 tablet.cc:1620] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:37.115023 12248 tablet.cc:1722] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:37.115525 12248 tablet.cc:1620] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:37.118053 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=e311ea2d8193486890fd3ee5f41e088f] alter complete (version 1)
I20240426 09:49:37.123026 7275 raft_consensus.cc:1203] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:37.178097 7013 raft_consensus.cc:1203] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:37.219710 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:37.224665 12244 tablet.cc:1722] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:37.224985 12244 tablet.cc:1620] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:37.226156 12245 tablet.cc:1722] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:37.226459 12245 tablet.cc:1620] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:37.227079 12248 tablet.cc:1722] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:37.227373 12248 tablet.cc:1620] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:37.230618 6740 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=e311ea2d8193486890fd3ee5f41e088f] alter complete (version 2)
I20240426 09:49:37.333194 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:37.339035 12244 tablet.cc:1722] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:37.339247 12245 tablet.cc:1722] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:37.339391 12248 tablet.cc:1722] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:37.345060 6740 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=e311ea2d8193486890fd3ee5f41e088f] alter complete (version 3)
I20240426 09:49:37.363847 11767 raft_consensus.cc:482] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:37.363973 11767 raft_consensus.cc:504] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:37.364519 11767 leader_election.cc:288] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:37.364980 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957533c35bae4299b35b77ba9a465eb3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:37.365130 9286 raft_consensus.cc:2454] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:37.365455 9351 leader_election.cc:302] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:37.365643 11767 raft_consensus.cc:2781] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:37.365691 11767 raft_consensus.cc:482] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:37.365725 11767 raft_consensus.cc:3037] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.366501 11767 raft_consensus.cc:504] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:37.366847 11767 leader_election.cc:288] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:37.367517 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957533c35bae4299b35b77ba9a465eb3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:37.367630 9286 raft_consensus.cc:3037] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.367897 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957533c35bae4299b35b77ba9a465eb3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:37.367969 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957533c35bae4299b35b77ba9a465eb3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:37.368017 9669 raft_consensus.cc:2454] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:37.368083 9672 raft_consensus.cc:3037] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.368614 9286 raft_consensus.cc:2454] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:37.368892 9672 raft_consensus.cc:2454] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:37.368887 9351 leader_election.cc:302] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:37.369078 11767 raft_consensus.cc:2781] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:37.369154 11767 raft_consensus.cc:686] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:37.369299 11767 consensus_queue.cc:237] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:37.371289 6737 catalog_manager.cc:5484] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.401872 9265 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:37.402211 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:37.403617 6993 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:37.406082 9661 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:37.407838 9529 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:37.408404 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:37.417250 9396 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:37.425249 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:37.447985 6737 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:37.448182 6737 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:37.451368 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet ce3730359fcf45c8b1264ac4ecbc05bb
I20240426 09:49:37.452064 9529 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3730359fcf45c8b1264ac4ecbc05bb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:37 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:37.452152 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3730359fcf45c8b1264ac4ecbc05bb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:37 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:37.452173 9265 tablet_service.cc:1514] Processing DeleteTablet for tablet ce3730359fcf45c8b1264ac4ecbc05bb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:37 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:37.452435 12288 tablet_replica.cc:330] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:37.452603 12287 tablet_replica.cc:330] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:37.452626 12288 raft_consensus.cc:2229] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:37.452759 12287 raft_consensus.cc:2229] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:37.452795 12288 raft_consensus.cc:2258] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:37.453054 12287 raft_consensus.cc:2258] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:37.453476 12288 ts_tablet_manager.cc:1875] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:37.453678 12289 tablet_replica.cc:330] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:37.453853 12289 raft_consensus.cc:2229] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:37.454003 12289 raft_consensus.cc:2258] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:37.454099 12287 ts_tablet_manager.cc:1875] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:37.454548 12289 ts_tablet_manager.cc:1875] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:37.456214 12288 ts_tablet_manager.cc:1888] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:37.456333 12288 log.cc:1196] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/ce3730359fcf45c8b1264ac4ecbc05bb
I20240426 09:49:37.456734 12288 ts_tablet_manager.cc:1909] T ce3730359fcf45c8b1264ac4ecbc05bb P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:37.456887 12289 ts_tablet_manager.cc:1888] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:37.457003 12289 log.cc:1196] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/ce3730359fcf45c8b1264ac4ecbc05bb
I20240426 09:49:37.457430 12289 ts_tablet_manager.cc:1909] T ce3730359fcf45c8b1264ac4ecbc05bb P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:37.458001 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet ce3730359fcf45c8b1264ac4ecbc05bb (table rebalancer_dynamic_table_new_name [id=e311ea2d8193486890fd3ee5f41e088f]) successfully deleted
I20240426 09:49:37.458118 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet ce3730359fcf45c8b1264ac4ecbc05bb (table rebalancer_dynamic_table_new_name [id=e311ea2d8193486890fd3ee5f41e088f]) successfully deleted
I20240426 09:49:37.464732 12287 ts_tablet_manager.cc:1888] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:37.464799 12287 log.cc:1196] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/ce3730359fcf45c8b1264ac4ecbc05bb
I20240426 09:49:37.465035 12287 ts_tablet_manager.cc:1909] T ce3730359fcf45c8b1264ac4ecbc05bb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:37.465636 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet ce3730359fcf45c8b1264ac4ecbc05bb (table rebalancer_dynamic_table_new_name [id=e311ea2d8193486890fd3ee5f41e088f]) successfully deleted
I20240426 09:49:37.507342 11767 consensus_queue.cc:1035] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.516548 11767 consensus_queue.cc:1035] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.550367 7142 raft_consensus.cc:1261] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:37.550736 11564 consensus_queue.cc:1035] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:37.553028 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:37.565568 9661 tablet_service.cc:1467] Processing CreateTablet for tablet eac28c5da80c4bf9a6bab059bef24a33 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b0656754f4094945bd3e45603172f6db]), partition=
I20240426 09:49:37.565560 9529 tablet_service.cc:1467] Processing CreateTablet for tablet eac28c5da80c4bf9a6bab059bef24a33 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b0656754f4094945bd3e45603172f6db]), partition=
I20240426 09:49:37.565841 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eac28c5da80c4bf9a6bab059bef24a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.565848 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eac28c5da80c4bf9a6bab059bef24a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.569092 9396 tablet_service.cc:1467] Processing CreateTablet for tablet eac28c5da80c4bf9a6bab059bef24a33 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b0656754f4094945bd3e45603172f6db]), partition=
I20240426 09:49:37.569314 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eac28c5da80c4bf9a6bab059bef24a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.572424 12356 tablet_bootstrap.cc:492] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:37.574308 12356 tablet_bootstrap.cc:654] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.574594 12358 tablet_bootstrap.cc:492] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:37.575758 9396 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:37.576088 12358 tablet_bootstrap.cc:654] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.577656 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:37.580323 12357 tablet_bootstrap.cc:492] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:37.581542 12357 tablet_bootstrap.cc:654] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.581972 9661 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:37.583245 12356 tablet_bootstrap.cc:492] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:37.583580 9265 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:37.584048 12356 ts_tablet_manager.cc:1367] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:49:37.584403 12357 tablet_bootstrap.cc:492] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:37.584488 12357 ts_tablet_manager.cc:1367] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:37.584542 12356 raft_consensus.cc:348] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.584658 12356 raft_consensus.cc:374] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.584717 12356 raft_consensus.cc:729] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.584833 12356 consensus_queue.cc:260] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.585175 12356 ts_tablet_manager.cc:1398] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:37.584946 12357 raft_consensus.cc:348] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.585594 12357 raft_consensus.cc:374] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.585680 12357 raft_consensus.cc:729] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.585863 12357 consensus_queue.cc:260] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.586308 12357 ts_tablet_manager.cc:1398] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:37.591591 12358 tablet_bootstrap.cc:492] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:37.592049 12358 ts_tablet_manager.cc:1367] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240426 09:49:37.592427 6993 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:37.592983 12358 raft_consensus.cc:348] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.593183 12358 raft_consensus.cc:374] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.593302 12358 raft_consensus.cc:729] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.593518 12358 consensus_queue.cc:260] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.595856 11939 raft_consensus.cc:482] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:37.595944 11939 raft_consensus.cc:504] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.596292 11939 leader_election.cc:288] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:37.596838 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:37.596882 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac28c5da80c4bf9a6bab059bef24a33" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:37.596993 9416 raft_consensus.cc:2454] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:37.597383 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac28c5da80c4bf9a6bab059bef24a33" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:37.597486 9672 raft_consensus.cc:2454] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:37.597756 9482 leader_election.cc:302] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:37.597993 11939 raft_consensus.cc:2781] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:37.598034 11939 raft_consensus.cc:482] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:37.598066 11939 raft_consensus.cc:3037] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.599092 11939 raft_consensus.cc:504] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.599407 11939 leader_election.cc:288] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:37.602756 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac28c5da80c4bf9a6bab059bef24a33" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:37.602907 9416 raft_consensus.cc:3037] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.603930 9416 raft_consensus.cc:2454] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:37.604269 12358 ts_tablet_manager.cc:1398] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20240426 09:49:37.604346 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eac28c5da80c4bf9a6bab059bef24a33" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:37.604455 9672 raft_consensus.cc:3037] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.605330 9672 raft_consensus.cc:2454] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:37.605607 9482 leader_election.cc:302] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:37.605965 12235 raft_consensus.cc:2781] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:37.606053 12235 raft_consensus.cc:686] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:37.606178 12235 consensus_queue.cc:237] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.608201 9529 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:37.610329 6880 raft_consensus.cc:1261] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:37.610844 11564 consensus_queue.cc:1035] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:37.617302 9548 raft_consensus.cc:1261] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:37.617631 11564 consensus_queue.cc:1035] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:37.618930 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:37.618940 6737 catalog_manager.cc:5484] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:37.649717 12235 consensus_queue.cc:1035] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.653261 10912 consensus_queue.cc:1035] T 08fa514a62a84c0e8068bfe23d175773 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.674363 9286 consensus_queue.cc:237] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:37.675597 7013 raft_consensus.cc:1261] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:37.675676 9416 raft_consensus.cc:1261] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:37.676059 12388 consensus_queue.cc:1035] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:37.676182 11842 consensus_queue.cc:1035] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:37.677954 11842 raft_consensus.cc:2932] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:37.680338 7011 raft_consensus.cc:2932] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:37.682721 6735 catalog_manager.cc:5484] T 5b5e68d9b3ce49d3aa398e85e139ddbf P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:37.684563 9416 raft_consensus.cc:2932] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:37.686075 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b5e68d9b3ce49d3aa398e85e139ddbf with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:37.687897 12397 tablet_replica.cc:330] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:37.688117 12397 raft_consensus.cc:2229] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:37.688385 12397 raft_consensus.cc:2258] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:37.688895 12397 ts_tablet_manager.cc:1875] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:37.691862 12397 ts_tablet_manager.cc:1888] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:37.692004 12397 log.cc:1196] T 5b5e68d9b3ce49d3aa398e85e139ddbf P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/5b5e68d9b3ce49d3aa398e85e139ddbf
I20240426 09:49:37.692716 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 5b5e68d9b3ce49d3aa398e85e139ddbf (table rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]) successfully deleted
I20240426 09:49:37.710503 7143 consensus_queue.cc:237] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:37.712100 9416 raft_consensus.cc:1261] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:37.712253 7079 consensus_peers.cc:485] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e010e7f54b54b04afe4e3b1df0c739f. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:37.712268 9548 raft_consensus.cc:1261] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:37.712479 12274 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:37.712656 12414 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:37.714741 12414 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.715132 9548 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.717602 6738 catalog_manager.cc:5484] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.717665 9416 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.727015 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:37.730914 9416 raft_consensus.cc:1261] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:37.731006 9672 raft_consensus.cc:1261] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:37.731444 12235 consensus_queue.cc:1035] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.731593 10912 consensus_queue.cc:1035] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.733943 6737 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet eac28c5da80c4bf9a6bab059bef24a33 (table rebalancer_dynamic_table [id=b0656754f4094945bd3e45603172f6db]). Expected version 1 got 0
I20240426 09:49:37.735101 12244 tablet.cc:1722] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:37.735402 12244 tablet.cc:1620] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:37.736945 6738 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet eac28c5da80c4bf9a6bab059bef24a33 (table rebalancer_dynamic_table [id=b0656754f4094945bd3e45603172f6db]). Expected version 1 got 0
I20240426 09:49:37.738034 6738 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet eac28c5da80c4bf9a6bab059bef24a33 (table rebalancer_dynamic_table [id=b0656754f4094945bd3e45603172f6db]). Expected version 1 got 0
I20240426 09:49:37.740295 12429 tablet.cc:1722] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:37.740619 12429 tablet.cc:1620] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:37.741989 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=b0656754f4094945bd3e45603172f6db] alter complete (version 1)
I20240426 09:49:37.743896 12435 tablet.cc:1722] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:37.744287 12435 tablet.cc:1620] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:37.745329 9672 consensus_queue.cc:237] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:37.747170 9548 raft_consensus.cc:1261] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:37.747222 9416 raft_consensus.cc:1261] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:37.747457 9616 consensus_peers.cc:485] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e306113f9874277a60468d9248d45ce. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:37.747675 12437 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:37.748493 11869 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:37.751036 12437 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.751405 9548 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.753898 6738 catalog_manager.cc:5484] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.755059 9416 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.775784 7143 consensus_queue.cc:237] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:37.777246 7013 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:37.777333 7077 consensus_peers.cc:485] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0362b7d68d9048bca47d96e948563d03. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:37.777644 7275 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:37.777638 12414 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.778012 12414 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.779764 11713 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:37.780273 7013 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:37.781790 7275 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:37.783042 6735 catalog_manager.cc:5484] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:37.800442 9548 consensus_queue.cc:237] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:37.801895 7013 raft_consensus.cc:1261] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:37.802007 9416 raft_consensus.cc:1261] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:37.802250 9483 consensus_peers.cc:485] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e3114093f8f24864b7184f0bd91fb401. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:37.802336 10912 consensus_queue.cc:1035] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:37.802507 12235 consensus_queue.cc:1035] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:37.804229 10912 raft_consensus.cc:2932] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.804651 7013 raft_consensus.cc:2932] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.805950 9416 raft_consensus.cc:2932] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.806737 6732 catalog_manager.cc:5484] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.826383 6880 consensus_queue.cc:237] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.827775 7143 raft_consensus.cc:1261] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:37.827800 6815 consensus_peers.cc:485] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5951b29e98cc4ff3a459dec654580ec0. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:37.827901 7275 raft_consensus.cc:1261] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:37.828225 12493 consensus_queue.cc:1035] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.828379 11818 consensus_queue.cc:1035] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.830135 12493 raft_consensus.cc:2932] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:37.830546 7143 raft_consensus.cc:2932] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:37.833508 6732 catalog_manager.cc:5484] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:37.833920 11767 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.837385 12091 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:37.837414 7275 raft_consensus.cc:2932] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:37.846907 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:37.852322 12244 tablet.cc:1722] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:37.852664 12244 tablet.cc:1620] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:37.852855 12435 tablet.cc:1722] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:37.853170 12435 tablet.cc:1620] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:37.853665 12429 tablet.cc:1722] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:37.853984 12429 tablet.cc:1620] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:37.858144 7013 consensus_queue.cc:237] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:37.859725 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=b0656754f4094945bd3e45603172f6db] alter complete (version 2)
I20240426 09:49:37.860131 7143 raft_consensus.cc:1261] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:37.860900 7275 raft_consensus.cc:1261] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:37.860975 6946 consensus_peers.cc:485] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be115ad80a4b42edbc6b2c630e54aedc. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:37.861083 12366 consensus_queue.cc:1035] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.861263 12076 consensus_queue.cc:1035] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.863592 12366 raft_consensus.cc:2932] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:37.864024 7275 raft_consensus.cc:2932] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:37.865577 7143 raft_consensus.cc:2932] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:37.866235 6738 catalog_manager.cc:5484] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:37.883620 7143 consensus_queue.cc:237] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:37.884896 7275 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:37.885124 7078 consensus_peers.cc:485] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ba744a156204d9e81db6ea113945203. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:37.885121 7013 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:37.885479 12414 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.885655 12274 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.887130 12414 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.887477 7275 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.889827 6735 catalog_manager.cc:5484] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.889863 7013 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:37.906981 7013 consensus_queue.cc:237] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.908370 7275 raft_consensus.cc:1261] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:37.908380 7143 raft_consensus.cc:1261] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:37.908412 6946 consensus_peers.cc:485] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5baaf8ce5a247e2975f8d6327349a26. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:37.908708 12076 consensus_queue.cc:1035] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.908872 12366 consensus_queue.cc:1035] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:37.910743 12076 raft_consensus.cc:2932] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:37.911192 7275 raft_consensus.cc:2932] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:37.913030 7143 raft_consensus.cc:2932] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:37.913612 6738 catalog_manager.cc:5484] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:37.932437 7143 consensus_queue.cc:237] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:37.933692 9286 raft_consensus.cc:1261] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:37.933733 9672 raft_consensus.cc:1261] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:37.933930 7079 consensus_peers.cc:485] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0779912d7a04b4c95dd1db6f9c4d6fd. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:37.934062 12535 consensus_queue.cc:1035] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:37.934240 12274 consensus_queue.cc:1035] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:37.935022 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:37.940886 9529 tablet_service.cc:1467] Processing CreateTablet for tablet cfe16e4114924bda8c560d4926a63923 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:37.940965 6862 tablet_service.cc:1467] Processing CreateTablet for tablet cfe16e4114924bda8c560d4926a63923 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:37.941232 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe16e4114924bda8c560d4926a63923. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.941232 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe16e4114924bda8c560d4926a63923. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.941478 9265 tablet_service.cc:1467] Processing CreateTablet for tablet cfe16e4114924bda8c560d4926a63923 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:37.941733 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe16e4114924bda8c560d4926a63923. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.942747 9528 tablet_service.cc:1467] Processing CreateTablet for tablet 6983bc4d17d44f998a749f7ec95bb305 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:37.943567 12535 raft_consensus.cc:2932] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.944273 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 67e3f93c0a2944e38b7542d1439c907e (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:37.944491 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e3f93c0a2944e38b7542d1439c907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.944561 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 67e3f93c0a2944e38b7542d1439c907e (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:37.944818 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e3f93c0a2944e38b7542d1439c907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.945513 9286 raft_consensus.cc:2932] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.945770 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 6983bc4d17d44f998a749f7ec95bb305 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:37.946015 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6983bc4d17d44f998a749f7ec95bb305. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.946949 12356 tablet_bootstrap.cc:492] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:37.947852 12356 tablet_bootstrap.cc:654] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.948884 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 6983bc4d17d44f998a749f7ec95bb305 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:37.948962 6732 catalog_manager.cc:5484] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.949154 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6983bc4d17d44f998a749f7ec95bb305. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.949407 9672 raft_consensus.cc:2932] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:37.950542 12574 tablet_bootstrap.cc:492] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:37.951157 12358 tablet_bootstrap.cc:492] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:37.952155 12574 tablet_bootstrap.cc:654] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.952196 12358 tablet_bootstrap.cc:654] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.952677 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6983bc4d17d44f998a749f7ec95bb305. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.953150 12573 tablet_bootstrap.cc:492] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:37.954077 12573 tablet_bootstrap.cc:654] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.955443 12357 tablet_bootstrap.cc:492] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:37.956295 12357 tablet_bootstrap.cc:654] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.959663 12356 tablet_bootstrap.cc:492] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:37.959746 12356 ts_tablet_manager.cc:1367] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240426 09:49:37.960042 12358 tablet_bootstrap.cc:492] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:37.960130 12358 ts_tablet_manager.cc:1367] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:37.960148 12356 raft_consensus.cc:348] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:37.960240 12356 raft_consensus.cc:374] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.960265 12356 raft_consensus.cc:729] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.960366 12356 consensus_queue.cc:260] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:37.960638 12356 ts_tablet_manager.cc:1398] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:37.960618 12358 raft_consensus.cc:348] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.960726 12358 raft_consensus.cc:374] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.960762 12358 raft_consensus.cc:729] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.960780 12356 tablet_bootstrap.cc:492] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:37.960904 12358 consensus_queue.cc:260] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.961174 12358 ts_tablet_manager.cc:1398] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:37.961848 12356 tablet_bootstrap.cc:654] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.962064 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 67e3f93c0a2944e38b7542d1439c907e (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:37.962231 12573 tablet_bootstrap.cc:492] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:37.962319 12573 ts_tablet_manager.cc:1367] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:37.962332 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e3f93c0a2944e38b7542d1439c907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:37.963398 12574 tablet_bootstrap.cc:492] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:37.963476 12574 ts_tablet_manager.cc:1367] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:37.963902 12574 raft_consensus.cc:348] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:37.964008 12574 raft_consensus.cc:374] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.964036 12574 raft_consensus.cc:729] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.964259 12573 raft_consensus.cc:348] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:37.964362 12573 raft_consensus.cc:374] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.964390 12573 raft_consensus.cc:729] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.964519 12574 consensus_queue.cc:260] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:37.964782 12574 ts_tablet_manager.cc:1398] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:37.964982 12573 consensus_queue.cc:260] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:37.965180 12574 tablet_bootstrap.cc:492] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:37.965412 12573 ts_tablet_manager.cc:1398] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:37.966034 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:37.966306 12573 tablet_bootstrap.cc:492] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:37.967088 12574 tablet_bootstrap.cc:654] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.968101 12573 tablet_bootstrap.cc:654] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.970579 12357 tablet_bootstrap.cc:492] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:37.970695 12357 ts_tablet_manager.cc:1367] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240426 09:49:37.971248 12357 raft_consensus.cc:348] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.971361 12357 raft_consensus.cc:374] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.971395 12357 raft_consensus.cc:729] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.971537 12357 consensus_queue.cc:260] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.971662 12573 tablet_bootstrap.cc:492] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:37.971783 12573 ts_tablet_manager.cc:1367] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:37.971863 12357 ts_tablet_manager.cc:1398] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:37.972198 12573 raft_consensus.cc:348] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.972323 12573 raft_consensus.cc:374] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.972400 12573 raft_consensus.cc:729] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.972712 12573 consensus_queue.cc:260] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.973029 12573 ts_tablet_manager.cc:1398] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:37.975464 12244 tablet.cc:1722] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:37.976325 12356 tablet_bootstrap.cc:492] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:37.976414 12356 ts_tablet_manager.cc:1367] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240426 09:49:37.976542 12574 tablet_bootstrap.cc:492] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:37.976752 12574 ts_tablet_manager.cc:1367] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:49:37.976902 12356 raft_consensus.cc:348] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.977006 12356 raft_consensus.cc:374] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.977052 12356 raft_consensus.cc:729] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.977034 12435 tablet.cc:1722] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:37.977149 12356 consensus_queue.cc:260] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.977236 12574 raft_consensus.cc:348] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.977330 12574 raft_consensus.cc:374] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.977358 12574 raft_consensus.cc:729] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.977408 12356 ts_tablet_manager.cc:1398] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:37.977497 12574 consensus_queue.cc:260] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:37.977728 12574 ts_tablet_manager.cc:1398] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:37.979036 12574 tablet_bootstrap.cc:492] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:37.980063 12574 tablet_bootstrap.cc:654] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:37.980934 12429 tablet.cc:1722] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:37.986061 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=b0656754f4094945bd3e45603172f6db] alter complete (version 3)
I20240426 09:49:37.987751 12574 tablet_bootstrap.cc:492] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:37.987849 12574 ts_tablet_manager.cc:1367] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:37.988196 12437 raft_consensus.cc:482] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:37.988282 12437 raft_consensus.cc:504] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.988665 12437 leader_election.cc:288] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:37.988665 12574 raft_consensus.cc:348] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.988765 12574 raft_consensus.cc:374] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:37.988801 12574 raft_consensus.cc:729] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:37.989073 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e3f93c0a2944e38b7542d1439c907e" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:37.989120 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e3f93c0a2944e38b7542d1439c907e" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:37.989199 6880 raft_consensus.cc:2454] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:37.989876 9616 leader_election.cc:302] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:37.989739 12574 consensus_queue.cc:260] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.990021 9286 raft_consensus.cc:2454] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:37.990082 12437 raft_consensus.cc:2781] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:37.990135 12437 raft_consensus.cc:482] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:37.990176 12437 raft_consensus.cc:3037] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.990332 12574 ts_tablet_manager.cc:1398] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:37.991255 12437 raft_consensus.cc:504] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.992276 12437 leader_election.cc:288] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:37.992661 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e3f93c0a2944e38b7542d1439c907e" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:37.992774 9286 raft_consensus.cc:3037] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.993728 9286 raft_consensus.cc:2454] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:37.994107 9615 leader_election.cc:302] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:37.994606 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67e3f93c0a2944e38b7542d1439c907e" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:37.994726 6880 raft_consensus.cc:3037] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:37.995800 6880 raft_consensus.cc:2454] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:37.996452 12437 raft_consensus.cc:2781] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:37.996542 12437 raft_consensus.cc:686] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:37.996672 12437 consensus_queue.cc:237] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:37.999363 6735 catalog_manager.cc:5484] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:38.008456 9548 consensus_queue.cc:237] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.009750 7143 raft_consensus.cc:1261] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:38.009830 9672 raft_consensus.cc:1261] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:38.009979 9482 consensus_peers.cc:485] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd67a40e0b834a83b41259ee4da3f3db. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:38.010195 12476 consensus_queue.cc:1035] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:38.010396 12235 consensus_queue.cc:1035] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:38.012136 12235 raft_consensus.cc:2932] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:38.012554 9672 raft_consensus.cc:2932] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:38.014370 7143 raft_consensus.cc:2932] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:38.015012 6732 catalog_manager.cc:5484] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:38.065749 12476 raft_consensus.cc:482] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:38.065922 12476 raft_consensus.cc:504] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:38.066387 12476 leader_election.cc:288] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:38.066686 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe16e4114924bda8c560d4926a63923" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:38.066844 9286 raft_consensus.cc:2454] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:38.066859 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe16e4114924bda8c560d4926a63923" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:38.067016 6880 raft_consensus.cc:2454] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:38.067086 9483 leader_election.cc:302] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:38.067257 12476 raft_consensus.cc:2781] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:38.067308 12476 raft_consensus.cc:482] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:38.067354 12476 raft_consensus.cc:3037] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.068249 12476 raft_consensus.cc:504] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:38.068624 12476 leader_election.cc:288] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:38.069161 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe16e4114924bda8c560d4926a63923" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:38.069160 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfe16e4114924bda8c560d4926a63923" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:38.069270 6880 raft_consensus.cc:3037] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.069270 9286 raft_consensus.cc:3037] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.070187 9286 raft_consensus.cc:2454] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:38.070187 6880 raft_consensus.cc:2454] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:38.070730 9484 leader_election.cc:302] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:38.070909 12476 raft_consensus.cc:2781] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:38.071005 12476 raft_consensus.cc:686] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:38.071134 12476 consensus_queue.cc:237] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:38.073251 6739 catalog_manager.cc:5484] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:38.089598 6737 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:38.089718 6737 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:38.091607 6737 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet eac28c5da80c4bf9a6bab059bef24a33
I20240426 09:49:38.092203 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet eac28c5da80c4bf9a6bab059bef24a33 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:38 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:38.092245 9528 tablet_service.cc:1514] Processing DeleteTablet for tablet eac28c5da80c4bf9a6bab059bef24a33 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:38 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:38.092214 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet eac28c5da80c4bf9a6bab059bef24a33 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:38 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:38.092417 12397 tablet_replica.cc:330] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:38.092563 12397 raft_consensus.cc:2229] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:38.092692 12397 raft_consensus.cc:2258] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:38.093250 12647 tablet_replica.cc:330] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:38.093355 12397 ts_tablet_manager.cc:1875] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:38.093469 12646 tablet_replica.cc:330] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:38.093587 12646 raft_consensus.cc:2229] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:38.093704 12646 raft_consensus.cc:2258] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:38.093887 12647 raft_consensus.cc:2229] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:38.094076 12647 raft_consensus.cc:2258] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:38.094868 12646 ts_tablet_manager.cc:1875] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:38.095605 12397 ts_tablet_manager.cc:1888] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:38.095665 12397 log.cc:1196] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/eac28c5da80c4bf9a6bab059bef24a33
I20240426 09:49:38.095954 12397 ts_tablet_manager.cc:1909] T eac28c5da80c4bf9a6bab059bef24a33 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:38.096628 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet eac28c5da80c4bf9a6bab059bef24a33 (table rebalancer_dynamic_table_new_name [id=b0656754f4094945bd3e45603172f6db]) successfully deleted
I20240426 09:49:38.097038 12646 ts_tablet_manager.cc:1888] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:38.097105 12646 log.cc:1196] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/eac28c5da80c4bf9a6bab059bef24a33
I20240426 09:49:38.097366 12646 ts_tablet_manager.cc:1909] T eac28c5da80c4bf9a6bab059bef24a33 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:38.097960 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet eac28c5da80c4bf9a6bab059bef24a33 (table rebalancer_dynamic_table_new_name [id=b0656754f4094945bd3e45603172f6db]) successfully deleted
I20240426 09:49:38.098294 12647 ts_tablet_manager.cc:1875] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:38.100024 12647 ts_tablet_manager.cc:1888] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:38.100085 12647 log.cc:1196] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/eac28c5da80c4bf9a6bab059bef24a33
I20240426 09:49:38.100301 12647 ts_tablet_manager.cc:1909] T eac28c5da80c4bf9a6bab059bef24a33 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:38.101012 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet eac28c5da80c4bf9a6bab059bef24a33 (table rebalancer_dynamic_table_new_name [id=b0656754f4094945bd3e45603172f6db]) successfully deleted
I20240426 09:49:38.128335 12391 raft_consensus.cc:482] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:38.128449 12391 raft_consensus.cc:504] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:38.128875 12391 leader_election.cc:288] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:38.129544 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6983bc4d17d44f998a749f7ec95bb305" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:38.129691 9416 raft_consensus.cc:2454] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:38.129949 9219 leader_election.cc:302] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:38.130173 12391 raft_consensus.cc:2781] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:38.130236 12391 raft_consensus.cc:482] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:38.130278 12391 raft_consensus.cc:3037] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.131680 12391 raft_consensus.cc:504] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:38.132035 12391 leader_election.cc:288] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:38.132416 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6983bc4d17d44f998a749f7ec95bb305" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:38.132512 9416 raft_consensus.cc:3037] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.133291 9416 raft_consensus.cc:2454] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:38.133580 9219 leader_election.cc:302] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:38.133582 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6983bc4d17d44f998a749f7ec95bb305" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:38.133658 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6983bc4d17d44f998a749f7ec95bb305" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:38.133705 9548 raft_consensus.cc:2454] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:38.133796 12391 raft_consensus.cc:2781] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:38.133873 12391 raft_consensus.cc:686] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:38.134020 12391 consensus_queue.cc:237] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:38.135716 6737 catalog_manager.cc:5484] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:38.153494 6993 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:38.154070 9265 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:38.162936 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:38.167539 9528 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:38.173765 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:38.183149 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:38.184736 9396 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240426 09:49:38.185917 12663 ts_tablet_manager.cc:902] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:38.186373 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:38.186957 12663 tablet_copy_client.cc:308] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:49:38.188011 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet e3114093f8f24864b7184f0bd91fb401 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:57057)
I20240426 09:49:38.188128 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet e3114093f8f24864b7184f0bd91fb401 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:57057: session id = a692602e1a2e4794877e6fb92a7b9962-e3114093f8f24864b7184f0bd91fb401
I20240426 09:49:38.189429 9569 tablet_copy_source_session.cc:205] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.190441 12663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3114093f8f24864b7184f0bd91fb401. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.192715 12663 tablet_copy_client.cc:783] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.192852 12663 tablet_copy_client.cc:647] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.197806 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:38.205804 12663 tablet_copy_client.cc:523] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.207558 12663 tablet_bootstrap.cc:492] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:38.208369 9661 tablet_service.cc:1467] Processing CreateTablet for tablet b77432669c7a4ba5939a4ea600af435f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8eff96e3606b439f8f8e34fc1457a420]), partition=
I20240426 09:49:38.208632 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b77432669c7a4ba5939a4ea600af435f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.209110 7123 tablet_service.cc:1467] Processing CreateTablet for tablet b77432669c7a4ba5939a4ea600af435f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8eff96e3606b439f8f8e34fc1457a420]), partition=
I20240426 09:49:38.209314 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b77432669c7a4ba5939a4ea600af435f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.210740 12357 tablet_bootstrap.cc:492] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:38.211575 12669 tablet_bootstrap.cc:492] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:38.211717 12357 tablet_bootstrap.cc:654] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.212550 9396 tablet_service.cc:1467] Processing CreateTablet for tablet b77432669c7a4ba5939a4ea600af435f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=8eff96e3606b439f8f8e34fc1457a420]), partition=
I20240426 09:49:38.212751 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b77432669c7a4ba5939a4ea600af435f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.214665 12358 tablet_bootstrap.cc:492] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:38.215128 12669 tablet_bootstrap.cc:654] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.215268 12357 tablet_bootstrap.cc:492] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:38.215342 12357 ts_tablet_manager.cc:1367] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:38.215569 12358 tablet_bootstrap.cc:654] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.215701 12357 raft_consensus.cc:348] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.215787 12357 raft_consensus.cc:374] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.215812 12357 raft_consensus.cc:729] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.215914 12357 consensus_queue.cc:260] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.216128 12357 ts_tablet_manager.cc:1398] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.216470 12358 tablet_bootstrap.cc:492] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:38.216542 12358 ts_tablet_manager.cc:1367] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:38.216909 12358 raft_consensus.cc:348] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.217237 12358 raft_consensus.cc:374] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.217278 12358 raft_consensus.cc:729] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.217394 12358 consensus_queue.cc:260] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.217670 12358 ts_tablet_manager.cc:1398] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:38.218010 12669 tablet_bootstrap.cc:492] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:38.218118 12669 ts_tablet_manager.cc:1367] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:38.218298 12663 tablet_bootstrap.cc:492] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.218983 12091 raft_consensus.cc:482] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:38.219074 12091 raft_consensus.cc:504] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.219122 12663 tablet_bootstrap.cc:492] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:38.219218 12669 raft_consensus.cc:348] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.219286 12663 ts_tablet_manager.cc:1367] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.003s
I20240426 09:49:38.219327 12669 raft_consensus.cc:374] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.219403 12669 raft_consensus.cc:729] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.219521 12091 leader_election.cc:288] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:38.219843 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77432669c7a4ba5939a4ea600af435f" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:38.219971 9672 raft_consensus.cc:2454] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:38.220002 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77432669c7a4ba5939a4ea600af435f" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:38.220270 9350 leader_election.cc:302] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:38.220031 12669 consensus_queue.cc:260] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.220480 7143 raft_consensus.cc:2454] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:38.220600 12663 raft_consensus.cc:348] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.220716 12663 raft_consensus.cc:729] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.221071 12091 raft_consensus.cc:2781] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:38.221128 12091 raft_consensus.cc:482] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:38.221163 12091 raft_consensus.cc:3037] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.220852 12663 consensus_queue.cc:260] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.221891 12663 ts_tablet_manager.cc:1398] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240426 09:49:38.222137 12091 raft_consensus.cc:504] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.222433 12091 leader_election.cc:288] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:38.222780 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-e3114093f8f24864b7184f0bd91fb401 received from {username='slave'} at 127.0.99.133:57057
I20240426 09:49:38.222884 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-e3114093f8f24864b7184f0bd91fb401 on tablet e3114093f8f24864b7184f0bd91fb401 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:38.223083 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77432669c7a4ba5939a4ea600af435f" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:38.223085 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b77432669c7a4ba5939a4ea600af435f" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:38.223177 9672 raft_consensus.cc:3037] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.223178 7143 raft_consensus.cc:3037] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.224100 7143 raft_consensus.cc:2454] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:38.224181 9672 raft_consensus.cc:2454] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:38.225170 9350 leader_election.cc:302] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:38.225317 12091 raft_consensus.cc:2781] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:38.225392 12091 raft_consensus.cc:686] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:38.225512 12091 consensus_queue.cc:237] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.225672 12669 ts_tablet_manager.cc:1398] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:38.227723 6731 catalog_manager.cc:5484] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:38.238502 12668 ts_tablet_manager.cc:902] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:38.239687 12668 tablet_copy_client.cc:308] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:38.243166 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 5e010e7f54b54b04afe4e3b1df0c739f from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:38167)
I20240426 09:49:38.243316 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 5e010e7f54b54b04afe4e3b1df0c739f from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:38167: session id = e3394a4be9ea4def8aa39b333c82c5ff-5e010e7f54b54b04afe4e3b1df0c739f
I20240426 09:49:38.244210 12679 ts_tablet_manager.cc:902] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:38.244354 7164 tablet_copy_source_session.cc:205] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.244693 12679 tablet_copy_client.cc:308] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:38.245143 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 5951b29e98cc4ff3a459dec654580ec0 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:56221)
I20240426 09:49:38.245244 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 5951b29e98cc4ff3a459dec654580ec0 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:56221: session id = 558c4ecd0ece460d9591aa7a3c799d19-5951b29e98cc4ff3a459dec654580ec0
I20240426 09:49:38.245256 12668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e010e7f54b54b04afe4e3b1df0c739f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.246205 6902 tablet_copy_source_session.cc:205] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.246990 12679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5951b29e98cc4ff3a459dec654580ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.247530 12668 tablet_copy_client.cc:783] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.247691 12668 tablet_copy_client.cc:647] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.248741 12668 tablet_copy_client.cc:523] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.248924 12679 tablet_copy_client.cc:783] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.249050 12679 tablet_copy_client.cc:647] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.250401 12668 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:38.251888 12679 tablet_copy_client.cc:523] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.253571 12679 tablet_bootstrap.cc:492] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:38.254956 12668 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.255190 12668 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:49:38.255385 12668 ts_tablet_manager.cc:1367] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20240426 09:49:38.255813 12668 raft_consensus.cc:348] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:38.255924 12668 raft_consensus.cc:729] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.256072 12668 consensus_queue.cc:260] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:38.256449 12668 ts_tablet_manager.cc:1398] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.263103 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-5e010e7f54b54b04afe4e3b1df0c739f received from {username='slave'} at 127.0.99.132:38167
I20240426 09:49:38.263192 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-5e010e7f54b54b04afe4e3b1df0c739f on tablet 5e010e7f54b54b04afe4e3b1df0c739f with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:38.275156 12679 tablet_bootstrap.cc:492] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.275492 12679 tablet_bootstrap.cc:492] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:38.275696 12679 ts_tablet_manager.cc:1367] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.022s user 0.021s sys 0.000s
I20240426 09:49:38.276232 12679 raft_consensus.cc:348] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:38.276363 12679 raft_consensus.cc:729] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.276495 12679 consensus_queue.cc:260] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:38.276791 12679 ts_tablet_manager.cc:1398] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:38.277478 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-5951b29e98cc4ff3a459dec654580ec0 received from {username='slave'} at 127.0.99.136:56221
I20240426 09:49:38.277570 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-5951b29e98cc4ff3a459dec654580ec0 on tablet 5951b29e98cc4ff3a459dec654580ec0 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:38.330369 9265 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:38.331732 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:38.332995 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:38.335239 9396 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:38.336020 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:38.336993 12663 ts_tablet_manager.cc:902] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:38.339922 12663 tablet_copy_client.cc:308] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:38.340432 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 5ba744a156204d9e81db6ea113945203 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:54629)
I20240426 09:49:38.340544 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 5ba744a156204d9e81db6ea113945203 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:54629: session id = a692602e1a2e4794877e6fb92a7b9962-5ba744a156204d9e81db6ea113945203
I20240426 09:49:38.341589 7164 tablet_copy_source_session.cc:205] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.342505 12663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba744a156204d9e81db6ea113945203. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.343283 9672 raft_consensus.cc:1261] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:38.343869 7143 raft_consensus.cc:1261] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:38.344523 11767 consensus_queue.cc:1035] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.344825 11767 consensus_queue.cc:1035] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.346438 12663 tablet_copy_client.cc:783] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.346616 12663 tablet_copy_client.cc:647] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.350950 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:38.354338 12663 tablet_copy_client.cc:523] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.354856 12429 tablet.cc:1722] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:38.355031 6732 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet b77432669c7a4ba5939a4ea600af435f (table rebalancer_dynamic_table [id=8eff96e3606b439f8f8e34fc1457a420]). Expected version 1 got 0
I20240426 09:49:38.355186 12429 tablet.cc:1620] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:38.355854 12663 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:38.360258 6737 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet b77432669c7a4ba5939a4ea600af435f (table rebalancer_dynamic_table [id=8eff96e3606b439f8f8e34fc1457a420]). Expected version 1 got 0
I20240426 09:49:38.361002 12668 ts_tablet_manager.cc:902] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:38.361471 12668 tablet_copy_client.cc:308] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:38.362954 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 8e306113f9874277a60468d9248d45ce from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:32841)
I20240426 09:49:38.363060 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 8e306113f9874277a60468d9248d45ce from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:32841: session id = e3394a4be9ea4def8aa39b333c82c5ff-8e306113f9874277a60468d9248d45ce
I20240426 09:49:38.363951 9685 tablet_copy_source_session.cc:205] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.364256 12435 tablet.cc:1722] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:38.364545 12435 tablet.cc:1620] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:38.365331 12668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e306113f9874277a60468d9248d45ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.366398 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=8eff96e3606b439f8f8e34fc1457a420] alter complete (version 1)
I20240426 09:49:38.367579 12668 tablet_copy_client.cc:783] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.367795 12668 tablet_copy_client.cc:647] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.368655 9528 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:38.369084 6732 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet b77432669c7a4ba5939a4ea600af435f (table rebalancer_dynamic_table [id=8eff96e3606b439f8f8e34fc1457a420]). Expected version 1 got 0
I20240426 09:49:38.372994 12668 tablet_copy_client.cc:523] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.374375 12668 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:38.376572 6993 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:38.378057 12668 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.378283 12668 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:49:38.378417 12668 ts_tablet_manager.cc:1367] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:49:38.378852 12668 raft_consensus.cc:348] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:38.378957 12668 raft_consensus.cc:729] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.379081 12668 consensus_queue.cc:260] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:38.379315 12668 ts_tablet_manager.cc:1398] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.380311 12737 ts_tablet_manager.cc:902] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:38.380810 12737 tablet_copy_client.cc:308] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:38.382424 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 0362b7d68d9048bca47d96e948563d03 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:49245)
I20240426 09:49:38.382522 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 0362b7d68d9048bca47d96e948563d03 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:49245: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-0362b7d68d9048bca47d96e948563d03
I20240426 09:49:38.383560 7164 tablet_copy_source_session.cc:205] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.383781 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-8e306113f9874277a60468d9248d45ce received from {username='slave'} at 127.0.99.132:32841
I20240426 09:49:38.383854 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-8e306113f9874277a60468d9248d45ce on tablet 8e306113f9874277a60468d9248d45ce with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:38.399603 12737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0362b7d68d9048bca47d96e948563d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.403941 12362 consensus_queue.cc:1035] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.405309 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:38.407838 12737 tablet_copy_client.cc:783] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.408032 12737 tablet_copy_client.cc:647] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.410979 12737 tablet_copy_client.cc:523] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.412701 12737 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:38.419782 12437 consensus_queue.cc:1035] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.427906 12663 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.428251 12663 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:38.428437 12663 ts_tablet_manager.cc:1367] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.073s user 0.025s sys 0.002s
I20240426 09:49:38.428877 12663 raft_consensus.cc:348] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.428989 12663 raft_consensus.cc:729] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.429122 12663 consensus_queue.cc:260] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.429891 12663 ts_tablet_manager.cc:1398] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.430369 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-5ba744a156204d9e81db6ea113945203 received from {username='slave'} at 127.0.99.133:54629
I20240426 09:49:38.430461 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-5ba744a156204d9e81db6ea113945203 on tablet 5ba744a156204d9e81db6ea113945203 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:38.435930 12757 ts_tablet_manager.cc:902] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:38.436360 12757 tablet_copy_client.cc:308] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:49:38.436869 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet cd67a40e0b834a83b41259ee4da3f3db from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:33967)
I20240426 09:49:38.436966 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet cd67a40e0b834a83b41259ee4da3f3db from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:33967: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-cd67a40e0b834a83b41259ee4da3f3db
I20240426 09:49:38.437850 9569 tablet_copy_source_session.cc:205] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.440276 6732 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet b77432669c7a4ba5939a4ea600af435f (table rebalancer_dynamic_table [id=8eff96e3606b439f8f8e34fc1457a420]). Expected version 1 got 0
I20240426 09:49:38.444198 12762 tablet.cc:1722] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:38.447400 12757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd67a40e0b834a83b41259ee4da3f3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.450510 12737 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.450511 12757 tablet_copy_client.cc:783] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.450686 12757 tablet_copy_client.cc:647] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.450997 12737 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:38.451203 12737 ts_tablet_manager.cc:1367] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.039s user 0.025s sys 0.001s
I20240426 09:49:38.451678 12737 raft_consensus.cc:348] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.451860 12737 raft_consensus.cc:729] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.452066 12737 consensus_queue.cc:260] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.452181 12763 ts_tablet_manager.cc:902] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:38.452901 12737 ts_tablet_manager.cc:1398] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:38.453612 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-0362b7d68d9048bca47d96e948563d03 received from {username='slave'} at 127.0.99.135:49245
I20240426 09:49:38.453699 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-0362b7d68d9048bca47d96e948563d03 on tablet 0362b7d68d9048bca47d96e948563d03 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:38.454213 12763 tablet_copy_client.cc:308] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:38.455016 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet be115ad80a4b42edbc6b2c630e54aedc from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:42099)
I20240426 09:49:38.444586 12762 tablet.cc:1620] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:38.455108 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet be115ad80a4b42edbc6b2c630e54aedc from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:42099: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-be115ad80a4b42edbc6b2c630e54aedc
I20240426 09:49:38.455999 7033 tablet_copy_source_session.cc:205] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.456300 12757 tablet_copy_client.cc:523] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.457617 12757 tablet_bootstrap.cc:492] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:38.460412 12763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be115ad80a4b42edbc6b2c630e54aedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.460945 12757 tablet_bootstrap.cc:492] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.461179 12757 tablet_bootstrap.cc:492] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:49:38.461347 12757 ts_tablet_manager.cc:1367] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:49:38.461808 12757 raft_consensus.cc:348] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.461927 12757 raft_consensus.cc:729] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.462069 12757 consensus_queue.cc:260] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.462702 12763 tablet_copy_client.cc:783] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.462872 12763 tablet_copy_client.cc:647] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.463280 12757 ts_tablet_manager.cc:1398] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:38.465282 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-cd67a40e0b834a83b41259ee4da3f3db received from {username='slave'} at 127.0.99.130:33967
I20240426 09:49:38.465338 12763 tablet_copy_client.cc:523] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.465353 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-cd67a40e0b834a83b41259ee4da3f3db on tablet cd67a40e0b834a83b41259ee4da3f3db with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:49:38.466599 12763 tablet_bootstrap.cc:492] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:38.471643 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:38.478232 12429 tablet.cc:1722] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:38.478536 12429 tablet.cc:1620] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:38.479014 12435 tablet.cc:1722] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:38.479300 12435 tablet.cc:1620] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:38.480165 12762 tablet.cc:1722] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:38.480446 12762 tablet.cc:1620] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:38.484987 12782 ts_tablet_manager.cc:902] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:38.485579 12782 tablet_copy_client.cc:308] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:38.485852 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=8eff96e3606b439f8f8e34fc1457a420] alter complete (version 2)
I20240426 09:49:38.486034 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet e5baaf8ce5a247e2975f8d6327349a26 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:38611)
I20240426 09:49:38.486138 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet e5baaf8ce5a247e2975f8d6327349a26 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:38611: session id = 3b41d87e7bfb419f8aa7500100129b6c-e5baaf8ce5a247e2975f8d6327349a26
I20240426 09:49:38.487129 7033 tablet_copy_source_session.cc:205] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.487991 12782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5baaf8ce5a247e2975f8d6327349a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.490029 12782 tablet_copy_client.cc:783] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.490154 12782 tablet_copy_client.cc:647] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.491009 12476 consensus_queue.cc:1035] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.494680 12476 consensus_queue.cc:1035] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.495687 12782 tablet_copy_client.cc:523] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.497045 12782 tablet_bootstrap.cc:492] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:38.500912 12763 tablet_bootstrap.cc:492] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.501394 12763 tablet_bootstrap.cc:492] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:38.501590 12763 ts_tablet_manager.cc:1367] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.003s
I20240426 09:49:38.502147 12763 raft_consensus.cc:348] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.502285 12763 raft_consensus.cc:729] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.502447 12763 consensus_queue.cc:260] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.502914 12763 ts_tablet_manager.cc:1398] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.503432 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-be115ad80a4b42edbc6b2c630e54aedc received from {username='slave'} at 127.0.99.135:42099
I20240426 09:49:38.503517 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-be115ad80a4b42edbc6b2c630e54aedc on tablet be115ad80a4b42edbc6b2c630e54aedc with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:38.522132 12782 tablet_bootstrap.cc:492] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.522555 12782 tablet_bootstrap.cc:492] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:38.522749 12782 ts_tablet_manager.cc:1367] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.026s user 0.019s sys 0.005s
I20240426 09:49:38.523315 12782 raft_consensus.cc:348] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:38.523449 12782 raft_consensus.cc:729] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.523649 12782 consensus_queue.cc:260] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:38.523943 12782 ts_tablet_manager.cc:1398] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:38.525133 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-e5baaf8ce5a247e2975f8d6327349a26 received from {username='slave'} at 127.0.99.134:38611
I20240426 09:49:38.525233 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-e5baaf8ce5a247e2975f8d6327349a26 on tablet e5baaf8ce5a247e2975f8d6327349a26 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:38.556690 12668 ts_tablet_manager.cc:902] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:38.557194 12668 tablet_copy_client.cc:308] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:38.557812 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet d0779912d7a04b4c95dd1db6f9c4d6fd from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:38167)
I20240426 09:49:38.557933 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet d0779912d7a04b4c95dd1db6f9c4d6fd from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:38167: session id = e3394a4be9ea4def8aa39b333c82c5ff-d0779912d7a04b4c95dd1db6f9c4d6fd
I20240426 09:49:38.559048 7164 tablet_copy_source_session.cc:205] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:38.561245 12668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0779912d7a04b4c95dd1db6f9c4d6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.563563 12668 tablet_copy_client.cc:783] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:38.563727 12668 tablet_copy_client.cc:647] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:38.566550 12668 tablet_copy_client.cc:523] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:38.568187 12668 tablet_bootstrap.cc:492] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:38.575909 6993 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:38.586583 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:38.590090 12668 tablet_bootstrap.cc:492] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:38.590374 12668 tablet_bootstrap.cc:492] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:49:38.590541 12668 ts_tablet_manager.cc:1367] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20240426 09:49:38.592694 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:38.595882 12668 raft_consensus.cc:348] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:38.596012 12668 raft_consensus.cc:729] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.596133 12668 consensus_queue.cc:260] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:38.596429 12668 ts_tablet_manager.cc:1398] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240426 09:49:38.597061 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-d0779912d7a04b4c95dd1db6f9c4d6fd received from {username='slave'} at 127.0.99.132:38167
I20240426 09:49:38.597126 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-d0779912d7a04b4c95dd1db6f9c4d6fd on tablet d0779912d7a04b4c95dd1db6f9c4d6fd with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:38.602828 7275 raft_consensus.cc:1203] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:38.603170 12429 tablet.cc:1722] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:38.604332 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:38.604840 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:38.605377 9265 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:38.608059 12435 tablet.cc:1722] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:38.614869 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=8eff96e3606b439f8f8e34fc1457a420] alter complete (version 3)
I20240426 09:49:38.620236 12762 tablet.cc:1722] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:38.623322 9528 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:38.636554 9396 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:38.637463 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:38.669548 6735 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:38.671880 9286 raft_consensus.cc:1203] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:38.676929 9396 tablet_service.cc:1467] Processing CreateTablet for tablet b32cacd1e7354933928c427f9646cafb (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:38.676929 9661 tablet_service.cc:1467] Processing CreateTablet for tablet b32cacd1e7354933928c427f9646cafb (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:38.677217 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b32cacd1e7354933928c427f9646cafb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.677224 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b32cacd1e7354933928c427f9646cafb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.679335 12358 tablet_bootstrap.cc:492] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:38.679702 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3624733f3444778b8656b3289d57dc (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:38.679877 9528 tablet_service.cc:1467] Processing CreateTablet for tablet b32cacd1e7354933928c427f9646cafb (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:38.680006 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3624733f3444778b8656b3289d57dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.680102 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b32cacd1e7354933928c427f9646cafb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.680280 12358 tablet_bootstrap.cc:654] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.680853 12357 tablet_bootstrap.cc:492] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:38.681787 12357 tablet_bootstrap.cc:654] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.682330 9527 tablet_service.cc:1467] Processing CreateTablet for tablet e64e5170d6bc4154b5ec869d535051c3 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:38.682541 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64e5170d6bc4154b5ec869d535051c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.683372 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3624733f3444778b8656b3289d57dc (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:38.683507 9266 tablet_service.cc:1467] Processing CreateTablet for tablet e64e5170d6bc4154b5ec869d535051c3 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:38.683609 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3624733f3444778b8656b3289d57dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.683735 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64e5170d6bc4154b5ec869d535051c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.683974 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3624733f3444778b8656b3289d57dc (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:38.684218 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3624733f3444778b8656b3289d57dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.684320 12358 tablet_bootstrap.cc:492] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:38.684393 12358 ts_tablet_manager.cc:1367] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:38.684834 12358 raft_consensus.cc:348] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.684942 12358 raft_consensus.cc:374] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.684971 12358 raft_consensus.cc:729] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.685082 12358 consensus_queue.cc:260] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.685370 12358 ts_tablet_manager.cc:1398] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.688498 12848 tablet_bootstrap.cc:492] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:38.688546 9396 tablet_service.cc:1467] Processing CreateTablet for tablet e64e5170d6bc4154b5ec869d535051c3 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:38.688779 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64e5170d6bc4154b5ec869d535051c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.689976 12848 tablet_bootstrap.cc:654] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.691407 12848 tablet_bootstrap.cc:492] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:38.691453 12357 tablet_bootstrap.cc:492] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:38.691546 12848 ts_tablet_manager.cc:1367] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:38.691557 12357 ts_tablet_manager.cc:1367] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:38.691735 12358 tablet_bootstrap.cc:492] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:38.691980 12848 raft_consensus.cc:348] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.691998 12357 raft_consensus.cc:348] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.692078 12848 raft_consensus.cc:374] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.692104 12848 raft_consensus.cc:729] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.692101 12357 raft_consensus.cc:374] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.692152 12357 raft_consensus.cc:729] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.692237 12848 consensus_queue.cc:260] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.692276 12357 consensus_queue.cc:260] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.692499 12848 ts_tablet_manager.cc:1398] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:38.692523 12357 ts_tablet_manager.cc:1398] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.692703 12358 tablet_bootstrap.cc:654] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.693851 12358 tablet_bootstrap.cc:492] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:38.693929 12358 ts_tablet_manager.cc:1367] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:38.694355 12358 raft_consensus.cc:348] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.694468 12358 raft_consensus.cc:374] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.694533 12358 raft_consensus.cc:729] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.694650 12358 consensus_queue.cc:260] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.694926 12358 ts_tablet_manager.cc:1398] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.695065 12847 tablet_bootstrap.cc:492] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:38.696337 12845 tablet_bootstrap.cc:492] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:38.696707 12847 tablet_bootstrap.cc:654] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.697535 12845 tablet_bootstrap.cc:654] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.700251 12845 tablet_bootstrap.cc:492] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:38.700402 12845 ts_tablet_manager.cc:1367] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:38.700922 12845 raft_consensus.cc:348] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.701052 12845 raft_consensus.cc:374] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.701120 12845 raft_consensus.cc:729] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.701275 12845 consensus_queue.cc:260] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.701658 12845 ts_tablet_manager.cc:1398] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.701862 12845 tablet_bootstrap.cc:492] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:38.702876 12845 tablet_bootstrap.cc:654] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.705463 12847 tablet_bootstrap.cc:492] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:38.705814 12847 ts_tablet_manager.cc:1367] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:38.706256 12847 raft_consensus.cc:348] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.706369 12847 raft_consensus.cc:374] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.706403 12847 raft_consensus.cc:729] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.706483 12391 consensus_queue.cc:1035] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.706511 12847 consensus_queue.cc:260] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.708336 12845 tablet_bootstrap.cc:492] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:38.708587 12845 ts_tablet_manager.cc:1367] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240426 09:49:38.709524 12845 raft_consensus.cc:348] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.709714 12845 raft_consensus.cc:374] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.710139 12845 raft_consensus.cc:729] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.710383 12845 consensus_queue.cc:260] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.710856 12845 ts_tablet_manager.cc:1398] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:38.711113 12845 tablet_bootstrap.cc:492] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:38.712512 12845 tablet_bootstrap.cc:654] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.712677 12847 ts_tablet_manager.cc:1398] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:38.712944 12847 tablet_bootstrap.cc:492] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:38.713469 9286 raft_consensus.cc:1203] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:38.715333 9549 raft_consensus.cc:3037] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.716500 12388 consensus_queue.cc:1035] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.717813 12845 tablet_bootstrap.cc:492] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:38.717931 12845 ts_tablet_manager.cc:1367] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240426 09:49:38.717998 6735 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:38.718155 6735 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:38.718421 12845 raft_consensus.cc:348] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.718537 12845 raft_consensus.cc:374] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.718581 12845 raft_consensus.cc:729] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.718691 12845 consensus_queue.cc:260] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.719019 12845 ts_tablet_manager.cc:1398] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:38.720283 6735 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet b77432669c7a4ba5939a4ea600af435f
I20240426 09:49:38.720960 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet b77432669c7a4ba5939a4ea600af435f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:38 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:38.720983 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet b77432669c7a4ba5939a4ea600af435f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:38 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:38.721338 12847 tablet_bootstrap.cc:654] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.721652 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet b77432669c7a4ba5939a4ea600af435f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:38 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:38.722154 12864 tablet_replica.cc:330] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:38.722167 12865 tablet_replica.cc:330] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:38.722309 12865 raft_consensus.cc:2229] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:38.722312 12864 raft_consensus.cc:2229] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:38.722481 12864 raft_consensus.cc:2258] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:38.722854 12865 raft_consensus.cc:2258] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:38.723217 12847 tablet_bootstrap.cc:492] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:38.723311 12847 ts_tablet_manager.cc:1367] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:38.723515 12865 ts_tablet_manager.cc:1875] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:38.723812 12847 raft_consensus.cc:348] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.723954 12847 raft_consensus.cc:374] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.724071 12847 raft_consensus.cc:729] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.724273 12847 consensus_queue.cc:260] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.724678 12847 ts_tablet_manager.cc:1398] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.725209 9672 raft_consensus.cc:1203] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:38.725263 12864 ts_tablet_manager.cc:1875] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:38.729889 9548 raft_consensus.cc:1203] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:38.731137 12864 ts_tablet_manager.cc:1888] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:38.731341 12865 ts_tablet_manager.cc:1888] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:38.732079 12863 tablet_replica.cc:330] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:38.732445 12863 raft_consensus.cc:2229] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:38.732658 12863 raft_consensus.cc:2258] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:38.733489 12863 ts_tablet_manager.cc:1875] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:38.733752 12865 log.cc:1196] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/b77432669c7a4ba5939a4ea600af435f
I20240426 09:49:38.733963 12864 log.cc:1196] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/b77432669c7a4ba5939a4ea600af435f
I20240426 09:49:38.734114 12865 ts_tablet_manager.cc:1909] T b77432669c7a4ba5939a4ea600af435f P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:38.735566 12864 ts_tablet_manager.cc:1909] T b77432669c7a4ba5939a4ea600af435f P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:38.736442 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet b77432669c7a4ba5939a4ea600af435f (table rebalancer_dynamic_table_new_name [id=8eff96e3606b439f8f8e34fc1457a420]) successfully deleted
I20240426 09:49:38.736636 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet b77432669c7a4ba5939a4ea600af435f (table rebalancer_dynamic_table_new_name [id=8eff96e3606b439f8f8e34fc1457a420]) successfully deleted
I20240426 09:49:38.738106 12863 ts_tablet_manager.cc:1888] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:38.738220 12863 log.cc:1196] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/b77432669c7a4ba5939a4ea600af435f
I20240426 09:49:38.738554 12863 ts_tablet_manager.cc:1909] T b77432669c7a4ba5939a4ea600af435f P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:38.739343 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet b77432669c7a4ba5939a4ea600af435f (table rebalancer_dynamic_table_new_name [id=8eff96e3606b439f8f8e34fc1457a420]) successfully deleted
I20240426 09:49:38.804898 11767 raft_consensus.cc:482] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:38.805042 11767 raft_consensus.cc:504] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.805488 11767 leader_election.cc:288] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:38.805825 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64e5170d6bc4154b5ec869d535051c3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:38.805869 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64e5170d6bc4154b5ec869d535051c3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:38.805987 9286 raft_consensus.cc:2454] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:38.805976 9549 raft_consensus.cc:2454] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:38.806264 9350 leader_election.cc:302] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:38.806432 11767 raft_consensus.cc:2781] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:38.806496 11767 raft_consensus.cc:482] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:38.806535 11767 raft_consensus.cc:3037] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.807551 11767 raft_consensus.cc:504] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.807930 11767 leader_election.cc:288] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:38.808266 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64e5170d6bc4154b5ec869d535051c3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:38.808364 9549 raft_consensus.cc:3037] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.809108 9549 raft_consensus.cc:2454] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:38.809324 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64e5170d6bc4154b5ec869d535051c3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:38.809374 9350 leader_election.cc:302] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:38.809435 9286 raft_consensus.cc:3037] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.809571 11767 raft_consensus.cc:2781] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:38.809664 11767 raft_consensus.cc:686] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:38.809796 11767 consensus_queue.cc:237] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:38.810338 9286 raft_consensus.cc:2454] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:38.811741 6735 catalog_manager.cc:5484] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:38.822427 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:38.827312 9396 tablet_service.cc:1467] Processing CreateTablet for tablet be2fccdea0cf4b7d856420bdfcd59521 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=24c3f75a65634dacac1ae29963c50238]), partition=
I20240426 09:49:38.827373 7123 tablet_service.cc:1467] Processing CreateTablet for tablet be2fccdea0cf4b7d856420bdfcd59521 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=24c3f75a65634dacac1ae29963c50238]), partition=
I20240426 09:49:38.827598 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2fccdea0cf4b7d856420bdfcd59521. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.827639 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2fccdea0cf4b7d856420bdfcd59521. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.829236 12358 tablet_bootstrap.cc:492] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:38.829455 9661 tablet_service.cc:1467] Processing CreateTablet for tablet be2fccdea0cf4b7d856420bdfcd59521 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=24c3f75a65634dacac1ae29963c50238]), partition=
I20240426 09:49:38.829702 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2fccdea0cf4b7d856420bdfcd59521. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:38.830219 12358 tablet_bootstrap.cc:654] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.831342 12358 tablet_bootstrap.cc:492] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:38.831429 12358 ts_tablet_manager.cc:1367] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:38.831924 12357 tablet_bootstrap.cc:492] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:38.831898 12358 raft_consensus.cc:348] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.832026 12358 raft_consensus.cc:374] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.832063 12358 raft_consensus.cc:729] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.832221 12358 consensus_queue.cc:260] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.832320 12898 tablet_bootstrap.cc:492] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:38.832463 12358 ts_tablet_manager.cc:1398] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:38.832751 12357 tablet_bootstrap.cc:654] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.833165 12898 tablet_bootstrap.cc:654] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:38.834030 12781 raft_consensus.cc:482] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:38.834146 12357 tablet_bootstrap.cc:492] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:38.834107 12781 raft_consensus.cc:504] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.834213 12357 ts_tablet_manager.cc:1367] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:38.834451 12781 leader_election.cc:288] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:38.834661 12357 raft_consensus.cc:348] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.834761 12357 raft_consensus.cc:374] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.834815 12357 raft_consensus.cc:729] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.834872 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2fccdea0cf4b7d856420bdfcd59521" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:38.834924 12357 consensus_queue.cc:260] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.835021 9672 raft_consensus.cc:2454] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:38.835139 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2fccdea0cf4b7d856420bdfcd59521" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:38.835266 12357 ts_tablet_manager.cc:1398] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20240426 09:49:38.835569 9350 leader_election.cc:341] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:38.835713 9350 leader_election.cc:302] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters: 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:38.835788 12898 tablet_bootstrap.cc:492] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:38.835920 12898 ts_tablet_manager.cc:1367] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:38.835968 12781 raft_consensus.cc:2781] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:38.836025 12781 raft_consensus.cc:482] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:38.836062 12781 raft_consensus.cc:3037] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.836568 12898 raft_consensus.cc:348] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.836652 12898 raft_consensus.cc:374] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:38.836671 12898 raft_consensus.cc:729] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:38.836776 12898 consensus_queue.cc:260] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.837013 12898 ts_tablet_manager.cc:1398] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:38.837312 12781 raft_consensus.cc:504] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.837618 12781 leader_election.cc:288] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:38.837873 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2fccdea0cf4b7d856420bdfcd59521" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:38.837963 9672 raft_consensus.cc:3037] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.837947 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2fccdea0cf4b7d856420bdfcd59521" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:38.838038 7143 raft_consensus.cc:3037] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.838840 9672 raft_consensus.cc:2454] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:38.838886 7143 raft_consensus.cc:2454] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:38.839177 9350 leader_election.cc:302] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:38.839462 12781 raft_consensus.cc:2781] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:38.839541 12781 raft_consensus.cc:686] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:38.839665 12781 consensus_queue.cc:237] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:38.841733 6737 catalog_manager.cc:5484] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:38.875121 7275 raft_consensus.cc:1203] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:38.883391 9416 raft_consensus.cc:1203] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:38.902402 12235 raft_consensus.cc:482] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:38.902518 12235 raft_consensus.cc:504] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.903007 12235 leader_election.cc:288] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:38.903378 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b32cacd1e7354933928c427f9646cafb" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:38.903496 9672 raft_consensus.cc:2454] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:38.903442 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b32cacd1e7354933928c427f9646cafb" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:38.903604 9416 raft_consensus.cc:2454] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:38.903774 9482 leader_election.cc:302] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:38.904055 12235 raft_consensus.cc:2781] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:38.904103 12235 raft_consensus.cc:482] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:38.904126 12235 raft_consensus.cc:3037] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.904881 12235 raft_consensus.cc:504] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.905143 12235 leader_election.cc:288] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:38.905436 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b32cacd1e7354933928c427f9646cafb" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:38.905537 9672 raft_consensus.cc:3037] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.905565 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b32cacd1e7354933928c427f9646cafb" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:38.905679 9416 raft_consensus.cc:3037] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.906406 9672 raft_consensus.cc:2454] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:38.906432 9416 raft_consensus.cc:2454] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:38.906688 9482 leader_election.cc:302] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:38.906951 12235 raft_consensus.cc:2781] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:38.907045 12235 raft_consensus.cc:686] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:38.907171 12235 consensus_queue.cc:237] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:38.908815 6738 catalog_manager.cc:5484] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:38.928494 12366 raft_consensus.cc:482] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:38.928645 12366 raft_consensus.cc:504] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.929064 12366 leader_election.cc:288] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:38.929528 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3624733f3444778b8656b3289d57dc" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:38.929528 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3624733f3444778b8656b3289d57dc" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:38.929670 9549 raft_consensus.cc:2454] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:38.929670 9286 raft_consensus.cc:2454] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:38.929929 6947 leader_election.cc:302] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:38.930141 12366 raft_consensus.cc:2781] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:38.930187 12366 raft_consensus.cc:482] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:38.930213 12366 raft_consensus.cc:3037] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.931144 12366 raft_consensus.cc:504] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.931532 12366 leader_election.cc:288] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:38.931871 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3624733f3444778b8656b3289d57dc" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:38.931982 9286 raft_consensus.cc:3037] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.932924 9286 raft_consensus.cc:2454] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:38.933669 6947 leader_election.cc:302] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:38.933836 12366 raft_consensus.cc:2781] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:38.933918 12366 raft_consensus.cc:686] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:38.934049 12366 consensus_queue.cc:237] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:38.935345 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3624733f3444778b8656b3289d57dc" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:38.935441 9549 raft_consensus.cc:3037] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:38.936071 6738 catalog_manager.cc:5484] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:38.936275 9549 raft_consensus.cc:2454] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:38.945947 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:38.950356 9672 raft_consensus.cc:1261] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:38.950881 11767 consensus_queue.cc:1035] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.957012 6738 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet be2fccdea0cf4b7d856420bdfcd59521 (table rebalancer_dynamic_table [id=24c3f75a65634dacac1ae29963c50238]). Expected version 1 got 0
I20240426 09:49:38.957271 7143 raft_consensus.cc:1261] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:38.959645 12429 tablet.cc:1722] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:38.959899 12435 tablet.cc:1722] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:38.960013 12429 tablet.cc:1620] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:38.960188 12435 tablet.cc:1620] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:38.964185 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=24c3f75a65634dacac1ae29963c50238] alter complete (version 1)
I20240426 09:49:38.968344 12781 consensus_queue.cc:1035] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:38.970599 7011 raft_consensus.cc:1203] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:38.976243 12739 mvcc.cc:205] Tried to move back new op lower bound from 7021055913776766976 to 7021055913328623616. Current Snapshot: MvccSnapshot[applied={T|T < 7021055913776766976}]
I20240426 09:49:38.976373 12762 tablet.cc:1722] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:38.976665 12762 tablet.cc:1620] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:38.979492 9265 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:38.980683 6993 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:38.981003 9396 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:38.987867 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:38.998616 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:38.999819 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:39.014943 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:39.021272 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:39.063686 9549 raft_consensus.cc:1203] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:39.069262 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:39.075289 12429 tablet.cc:1722] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:39.075628 12429 tablet.cc:1620] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:39.075614 12435 tablet.cc:1722] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:39.075876 12435 tablet.cc:1620] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:39.076416 12762 tablet.cc:1722] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:39.076681 12762 tablet.cc:1620] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:39.081614 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=24c3f75a65634dacac1ae29963c50238] alter complete (version 2)
I20240426 09:49:39.083853 7011 consensus_queue.cc:237] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.085060 9416 raft_consensus.cc:1261] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:39.085405 7275 raft_consensus.cc:1261] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:39.085821 12366 consensus_queue.cc:1035] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:39.086022 12905 consensus_queue.cc:1035] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:39.087424 12366 raft_consensus.cc:2932] T e5baaf8ce5a247e2975f8d6327349a26 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:39.087790 7274 raft_consensus.cc:2932] T e5baaf8ce5a247e2975f8d6327349a26 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:39.088250 9416 raft_consensus.cc:2932] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:39.091351 6735 catalog_manager.cc:5484] T e5baaf8ce5a247e2975f8d6327349a26 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:39.094198 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet e5baaf8ce5a247e2975f8d6327349a26 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:39.094458 12863 tablet_replica.cc:330] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:39.094619 12863 raft_consensus.cc:2229] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:39.094813 12863 raft_consensus.cc:2258] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:39.095409 12863 ts_tablet_manager.cc:1875] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:39.097093 12863 ts_tablet_manager.cc:1888] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:39.097147 12863 log.cc:1196] T e5baaf8ce5a247e2975f8d6327349a26 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/e5baaf8ce5a247e2975f8d6327349a26
I20240426 09:49:39.097605 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet e5baaf8ce5a247e2975f8d6327349a26 (table rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]) successfully deleted
I20240426 09:49:39.164006 7274 raft_consensus.cc:1203] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:39.183246 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:39.185899 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:39.192682 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:39.193075 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:39.195504 9265 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:39.196420 12429 tablet.cc:1722] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:39.203599 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=24c3f75a65634dacac1ae29963c50238] alter complete (version 3)
I20240426 09:49:39.205370 12435 tablet.cc:1722] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:39.211678 12762 tablet.cc:1722] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:39.215170 9396 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:39.221300 6993 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:39.224141 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:39.224673 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:39.233080 12931 consensus_queue.cc:1035] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:39.244446 12931 consensus_queue.cc:1035] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:39.281087 7143 consensus_queue.cc:237] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:39.282307 9286 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:39.282701 7274 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:39.282681 12903 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:39.283118 12903 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:39.284433 12920 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:39.284862 9286 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:39.285277 7274 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:39.287367 6740 catalog_manager.cc:5484] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:39.290678 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ba744a156204d9e81db6ea113945203 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:39.291060 13018 tablet_replica.cc:330] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:39.291263 13018 raft_consensus.cc:2229] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:39.291483 13018 raft_consensus.cc:2258] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:39.291976 13018 ts_tablet_manager.cc:1875] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:39.294070 13018 ts_tablet_manager.cc:1888] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:39.294184 13018 log.cc:1196] T 5ba744a156204d9e81db6ea113945203 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/5ba744a156204d9e81db6ea113945203
I20240426 09:49:39.294731 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 5ba744a156204d9e81db6ea113945203 (table rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]) successfully deleted
I20240426 09:49:39.302661 12235 consensus_queue.cc:1035] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:39.305097 6736 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:39.305222 6736 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:39.306145 12476 consensus_queue.cc:1035] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:39.309213 6736 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet be2fccdea0cf4b7d856420bdfcd59521
I20240426 09:49:39.309878 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet be2fccdea0cf4b7d856420bdfcd59521 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:39 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:39.309878 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet be2fccdea0cf4b7d856420bdfcd59521 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:39 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:39.310134 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet be2fccdea0cf4b7d856420bdfcd59521 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:39 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:39.310179 13029 tablet_replica.cc:330] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:39.310314 12863 tablet_replica.cc:330] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:39.310319 13029 raft_consensus.cc:2229] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:39.310426 12863 raft_consensus.cc:2229] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:39.310431 13029 raft_consensus.cc:2258] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:39.310544 12863 raft_consensus.cc:2258] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:39.310613 13028 tablet_replica.cc:330] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:39.310746 13028 raft_consensus.cc:2229] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:39.311251 13028 raft_consensus.cc:2258] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:39.311455 12863 ts_tablet_manager.cc:1875] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:39.311806 13028 ts_tablet_manager.cc:1875] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:39.313691 12863 ts_tablet_manager.cc:1888] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:39.313750 12863 log.cc:1196] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/be2fccdea0cf4b7d856420bdfcd59521
I20240426 09:49:39.313988 12863 ts_tablet_manager.cc:1909] T be2fccdea0cf4b7d856420bdfcd59521 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:39.314239 13028 ts_tablet_manager.cc:1888] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:39.314308 13028 log.cc:1196] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/be2fccdea0cf4b7d856420bdfcd59521
I20240426 09:49:39.314575 13028 ts_tablet_manager.cc:1909] T be2fccdea0cf4b7d856420bdfcd59521 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:39.314723 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet be2fccdea0cf4b7d856420bdfcd59521 (table rebalancer_dynamic_table_new_name [id=24c3f75a65634dacac1ae29963c50238]) successfully deleted
I20240426 09:49:39.315176 13029 ts_tablet_manager.cc:1875] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:39.315295 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet be2fccdea0cf4b7d856420bdfcd59521 (table rebalancer_dynamic_table_new_name [id=24c3f75a65634dacac1ae29963c50238]) successfully deleted
I20240426 09:49:39.317241 13029 ts_tablet_manager.cc:1888] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:39.317325 13029 log.cc:1196] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/be2fccdea0cf4b7d856420bdfcd59521
I20240426 09:49:39.317567 13029 ts_tablet_manager.cc:1909] T be2fccdea0cf4b7d856420bdfcd59521 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:39.318164 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet be2fccdea0cf4b7d856420bdfcd59521 (table rebalancer_dynamic_table_new_name [id=24c3f75a65634dacac1ae29963c50238]) successfully deleted
I20240426 09:49:39.381392 6993 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:39.383342 7123 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:39.387152 12366 consensus_queue.cc:1035] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:39.395057 12366 consensus_queue.cc:1035] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:39.400283 9265 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:39.403049 9396 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240426 09:49:39.403903 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:39.404033 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:39.417686 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:39.418749 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:39.425393 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1b192686984efcbb483742fe54850d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d7546910b6e642ddb6b1345ffabe7b4a]), partition=
I20240426 09:49:39.425645 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1b192686984efcbb483742fe54850d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.425972 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1b192686984efcbb483742fe54850d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d7546910b6e642ddb6b1345ffabe7b4a]), partition=
I20240426 09:49:39.426193 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1b192686984efcbb483742fe54850d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.427896 13074 tablet_bootstrap.cc:492] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:39.428550 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1b192686984efcbb483742fe54850d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d7546910b6e642ddb6b1345ffabe7b4a]), partition=
I20240426 09:49:39.428764 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1b192686984efcbb483742fe54850d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.428830 13074 tablet_bootstrap.cc:654] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.434968 13077 tablet_bootstrap.cc:492] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:39.435956 13077 tablet_bootstrap.cc:654] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.436771 13075 tablet_bootstrap.cc:492] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:39.437682 13075 tablet_bootstrap.cc:654] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.441638 13074 tablet_bootstrap.cc:492] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:39.441921 13074 ts_tablet_manager.cc:1367] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240426 09:49:39.442427 13074 raft_consensus.cc:348] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.442553 13074 raft_consensus.cc:374] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.442581 13074 raft_consensus.cc:729] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.442951 13074 consensus_queue.cc:260] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.443377 13074 ts_tablet_manager.cc:1398] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:39.448407 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:39.451412 13077 tablet_bootstrap.cc:492] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:39.451592 13077 ts_tablet_manager.cc:1367] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240426 09:49:39.452756 13077 raft_consensus.cc:348] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.452953 13077 raft_consensus.cc:374] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.453022 13077 raft_consensus.cc:729] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.453222 13077 consensus_queue.cc:260] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.453663 13077 ts_tablet_manager.cc:1398] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:39.456863 13075 tablet_bootstrap.cc:492] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:39.457034 13075 ts_tablet_manager.cc:1367] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240426 09:49:39.457509 13075 raft_consensus.cc:348] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.457618 13075 raft_consensus.cc:374] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.457645 13075 raft_consensus.cc:729] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.457791 13075 consensus_queue.cc:260] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.458155 13075 ts_tablet_manager.cc:1398] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:39.489320 7011 consensus_queue.cc:237] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:39.490531 7143 raft_consensus.cc:1261] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:39.490641 9549 raft_consensus.cc:1261] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:39.491007 12366 consensus_queue.cc:1035] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:39.491274 12905 consensus_queue.cc:1035] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:39.492679 12366 raft_consensus.cc:2932] T be115ad80a4b42edbc6b2c630e54aedc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:39.493067 7143 raft_consensus.cc:2932] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:39.494961 9549 raft_consensus.cc:2932] T be115ad80a4b42edbc6b2c630e54aedc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:39.496107 6738 catalog_manager.cc:5484] T be115ad80a4b42edbc6b2c630e54aedc P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:39.500036 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet be115ad80a4b42edbc6b2c630e54aedc with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:39.501116 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:39.503629 13104 tablet_replica.cc:330] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:39.503950 13104 raft_consensus.cc:2229] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:39.504179 13104 raft_consensus.cc:2258] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:39.504770 13104 ts_tablet_manager.cc:1875] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:39.505658 12931 raft_consensus.cc:482] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:39.505766 12931 raft_consensus.cc:504] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.506192 12931 leader_election.cc:288] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:39.506605 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1b192686984efcbb483742fe54850d" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:39.506757 9672 raft_consensus.cc:2454] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:39.507102 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1b192686984efcbb483742fe54850d" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:39.507251 9286 raft_consensus.cc:2454] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:39.508268 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 981b71e5e8b94db7b909ec0b7564152a (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:39.508538 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 981b71e5e8b94db7b909ec0b7564152a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.508755 9529 tablet_service.cc:1467] Processing CreateTablet for tablet feb385fea62d4d6fa355b49461c7a05a (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:39.508879 9660 tablet_service.cc:1467] Processing CreateTablet for tablet feb385fea62d4d6fa355b49461c7a05a (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:39.508955 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb385fea62d4d6fa355b49461c7a05a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.509097 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb385fea62d4d6fa355b49461c7a05a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.510932 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 981b71e5e8b94db7b909ec0b7564152a (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:39.511075 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 981b71e5e8b94db7b909ec0b7564152a (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:39.511181 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 981b71e5e8b94db7b909ec0b7564152a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.511340 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 981b71e5e8b94db7b909ec0b7564152a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.511709 9396 tablet_service.cc:1467] Processing CreateTablet for tablet feb385fea62d4d6fa355b49461c7a05a (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:39.511910 13111 tablet_bootstrap.cc:492] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:39.511957 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb385fea62d4d6fa355b49461c7a05a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.512843 13111 tablet_bootstrap.cc:654] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.513681 9265 tablet_service.cc:1467] Processing CreateTablet for tablet ab6450f48a36483883f8914075c224a8 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:39.513943 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6450f48a36483883f8914075c224a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.514078 13075 tablet_bootstrap.cc:492] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:39.514130 13111 tablet_bootstrap.cc:492] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:39.514196 13111 ts_tablet_manager.cc:1367] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:39.514627 13111 raft_consensus.cc:348] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:39.514708 13111 raft_consensus.cc:374] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.514735 13111 raft_consensus.cc:729] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.514870 13111 consensus_queue.cc:260] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:39.514973 13075 tablet_bootstrap.cc:654] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.514959 9529 tablet_service.cc:1467] Processing CreateTablet for tablet ab6450f48a36483883f8914075c224a8 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:39.515168 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6450f48a36483883f8914075c224a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.515233 13111 ts_tablet_manager.cc:1398] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:39.515379 13111 tablet_bootstrap.cc:492] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:39.516021 13077 tablet_bootstrap.cc:492] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:39.516386 13111 tablet_bootstrap.cc:654] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.516815 13077 tablet_bootstrap.cc:654] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.517416 13111 tablet_bootstrap.cc:492] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:39.517515 9397 tablet_service.cc:1467] Processing CreateTablet for tablet ab6450f48a36483883f8914075c224a8 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:39.517716 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6450f48a36483883f8914075c224a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:39.518150 13111 ts_tablet_manager.cc:1367] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:39.518525 13111 raft_consensus.cc:348] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.518615 13111 raft_consensus.cc:374] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.518857 13111 raft_consensus.cc:729] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.518985 13111 consensus_queue.cc:260] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.519227 13111 ts_tablet_manager.cc:1398] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:39.519753 13074 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:39.520071 13111 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:39.520548 13074 tablet_bootstrap.cc:654] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.520960 13111 tablet_bootstrap.cc:654] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.521553 9350 leader_election.cc:302] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:39.521742 12931 raft_consensus.cc:2781] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:39.521798 13074 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:39.521808 12931 raft_consensus.cc:482] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:39.521857 12931 raft_consensus.cc:3037] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.521873 13074 ts_tablet_manager.cc:1367] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:39.522369 13074 raft_consensus.cc:348] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.522455 13074 raft_consensus.cc:374] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.522485 13074 raft_consensus.cc:729] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.522588 13074 consensus_queue.cc:260] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.522784 12931 raft_consensus.cc:504] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.522861 13074 ts_tablet_manager.cc:1398] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:39.523157 12931 leader_election.cc:288] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:39.523483 13075 tablet_bootstrap.cc:492] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:39.523571 13075 ts_tablet_manager.cc:1367] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240426 09:49:39.523989 13075 raft_consensus.cc:348] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.524076 13075 raft_consensus.cc:374] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.524102 13075 raft_consensus.cc:729] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.524196 13075 consensus_queue.cc:260] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.524454 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1b192686984efcbb483742fe54850d" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:39.524497 13075 ts_tablet_manager.cc:1398] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:39.524578 9286 raft_consensus.cc:3037] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.524667 13075 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:39.525454 9286 raft_consensus.cc:2454] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:39.525538 13075 tablet_bootstrap.cc:654] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.525799 9351 leader_election.cc:302] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:39.522928 13077 tablet_bootstrap.cc:492] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:39.525974 12931 raft_consensus.cc:2781] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:39.526031 13077 ts_tablet_manager.cc:1367] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:39.526055 12931 raft_consensus.cc:686] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:39.526223 12931 consensus_queue.cc:237] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.526522 13077 raft_consensus.cc:348] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.526618 13077 raft_consensus.cc:374] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.526646 13077 raft_consensus.cc:729] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.526749 13077 consensus_queue.cc:260] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.527038 13077 ts_tablet_manager.cc:1398] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:39.527184 13077 tablet_bootstrap.cc:492] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:39.527395 13075 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:39.527460 13075 ts_tablet_manager.cc:1367] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:39.527925 13075 raft_consensus.cc:348] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.528002 13075 raft_consensus.cc:374] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.528029 13075 raft_consensus.cc:729] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.528115 13077 tablet_bootstrap.cc:654] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.528121 13075 consensus_queue.cc:260] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.528357 13075 ts_tablet_manager.cc:1398] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:39.524202 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1b192686984efcbb483742fe54850d" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:39.529428 9672 raft_consensus.cc:3037] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.529884 6738 catalog_manager.cc:5484] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:39.531528 13077 tablet_bootstrap.cc:492] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:39.531610 13077 ts_tablet_manager.cc:1367] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:49:39.533048 13077 raft_consensus.cc:348] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:39.533143 13077 raft_consensus.cc:374] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.533172 13077 raft_consensus.cc:729] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.533279 13077 consensus_queue.cc:260] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:39.533650 13077 ts_tablet_manager.cc:1398] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:39.534077 9672 raft_consensus.cc:2454] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:39.536676 13104 ts_tablet_manager.cc:1888] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:39.536782 13104 log.cc:1196] T be115ad80a4b42edbc6b2c630e54aedc P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/be115ad80a4b42edbc6b2c630e54aedc
I20240426 09:49:39.537348 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet be115ad80a4b42edbc6b2c630e54aedc (table rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]) successfully deleted
I20240426 09:49:39.537833 13113 tablet_bootstrap.cc:492] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:39.538767 13113 tablet_bootstrap.cc:654] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:39.540068 13111 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:39.540165 13111 ts_tablet_manager.cc:1367] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240426 09:49:39.540588 13111 raft_consensus.cc:348] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.540684 13111 raft_consensus.cc:374] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.540711 13111 raft_consensus.cc:729] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.540812 13111 consensus_queue.cc:260] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.541061 13111 ts_tablet_manager.cc:1398] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:39.547606 13113 tablet_bootstrap.cc:492] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:39.547714 13113 ts_tablet_manager.cc:1367] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240426 09:49:39.548168 13113 raft_consensus.cc:348] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:39.548285 13113 raft_consensus.cc:374] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:39.548377 13113 raft_consensus.cc:729] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:39.548565 13113 consensus_queue.cc:260] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:39.548885 13113 ts_tablet_manager.cc:1398] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:39.585471 12781 raft_consensus.cc:482] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:39.585582 12781 raft_consensus.cc:504] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.585966 12781 leader_election.cc:288] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:39.586495 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb385fea62d4d6fa355b49461c7a05a" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:39.586653 9548 raft_consensus.cc:2454] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:39.587273 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb385fea62d4d6fa355b49461c7a05a" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:39.587397 9672 raft_consensus.cc:2454] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:39.587797 9350 leader_election.cc:302] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:39.588109 12781 raft_consensus.cc:2781] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:39.588176 12781 raft_consensus.cc:482] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:39.588215 12781 raft_consensus.cc:3037] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.589898 12781 raft_consensus.cc:504] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.590293 12781 leader_election.cc:288] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:39.592209 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb385fea62d4d6fa355b49461c7a05a" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:39.592185 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb385fea62d4d6fa355b49461c7a05a" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:39.592350 9672 raft_consensus.cc:3037] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.592355 9549 raft_consensus.cc:3037] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.593116 9672 raft_consensus.cc:2454] T feb385fea62d4d6fa355b49461c7a05a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:39.593346 9549 raft_consensus.cc:2454] T feb385fea62d4d6fa355b49461c7a05a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:39.593394 9350 leader_election.cc:302] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:39.593549 12781 raft_consensus.cc:2781] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:39.593617 12781 raft_consensus.cc:686] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:39.593741 12781 consensus_queue.cc:237] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.596531 6736 catalog_manager.cc:5484] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:39.608402 13119 raft_consensus.cc:482] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:39.608515 13119 raft_consensus.cc:504] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.608902 13119 leader_election.cc:288] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:39.609354 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6450f48a36483883f8914075c224a8" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:39.609472 9549 raft_consensus.cc:2454] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:39.609449 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6450f48a36483883f8914075c224a8" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:39.609567 9417 raft_consensus.cc:2454] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:39.609920 9219 leader_election.cc:302] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:39.610066 13119 raft_consensus.cc:2781] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:39.610112 13119 raft_consensus.cc:482] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:39.610146 13119 raft_consensus.cc:3037] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.611346 13119 raft_consensus.cc:504] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.611689 13119 leader_election.cc:288] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:39.611964 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6450f48a36483883f8914075c224a8" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:39.612023 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6450f48a36483883f8914075c224a8" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:39.612078 9549 raft_consensus.cc:3037] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.612133 9417 raft_consensus.cc:3037] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.613093 9549 raft_consensus.cc:2454] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:39.613166 9417 raft_consensus.cc:2454] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:39.613381 9219 leader_election.cc:302] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:39.613582 13119 raft_consensus.cc:2781] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:39.613691 13119 raft_consensus.cc:686] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:39.613823 13119 consensus_queue.cc:237] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:39.614553 12920 raft_consensus.cc:482] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:39.614651 12920 raft_consensus.cc:504] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:39.615056 12920 leader_election.cc:288] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:39.615561 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "981b71e5e8b94db7b909ec0b7564152a" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:39.615572 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "981b71e5e8b94db7b909ec0b7564152a" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:39.615667 9549 raft_consensus.cc:2454] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:39.615692 9672 raft_consensus.cc:2454] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:39.616240 7077 leader_election.cc:302] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:39.616421 12920 raft_consensus.cc:2781] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:39.616482 12920 raft_consensus.cc:482] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:39.616518 12920 raft_consensus.cc:3037] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.616804 6734 catalog_manager.cc:5484] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:39.617537 12920 raft_consensus.cc:504] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:39.617862 12920 leader_election.cc:288] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:39.618258 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "981b71e5e8b94db7b909ec0b7564152a" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:39.618356 9549 raft_consensus.cc:3037] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.618335 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "981b71e5e8b94db7b909ec0b7564152a" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:39.618440 9672 raft_consensus.cc:3037] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:39.619477 9672 raft_consensus.cc:2454] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:39.619737 7077 leader_election.cc:302] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:39.620177 9549 raft_consensus.cc:2454] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:39.620533 12920 raft_consensus.cc:2781] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:39.620620 12920 raft_consensus.cc:686] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:39.620740 12920 consensus_queue.cc:237] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:39.622709 6734 catalog_manager.cc:5484] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:39.638051 9265 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:39.644752 6993 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:39.647347 9397 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:39.649602 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:39.651705 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:39.652108 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:39.657307 9529 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:39.661581 6734 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:39.671826 7123 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:39.673609 9672 raft_consensus.cc:1261] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:39.673709 9286 raft_consensus.cc:1261] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:39.674027 12781 consensus_queue.cc:1035] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:39.674302 12781 consensus_queue.cc:1035] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:39.676873 6738 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet 8d1b192686984efcbb483742fe54850d (table rebalancer_dynamic_table [id=d7546910b6e642ddb6b1345ffabe7b4a]). Expected version 1 got 0
I20240426 09:49:39.677695 6736 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet 8d1b192686984efcbb483742fe54850d (table rebalancer_dynamic_table [id=d7546910b6e642ddb6b1345ffabe7b4a]). Expected version 1 got 0
I20240426 09:49:39.687649 12429 tablet.cc:1722] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:39.687968 12429 tablet.cc:1620] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:39.688849 12435 tablet.cc:1722] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:39.689136 12435 tablet.cc:1620] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:39.694440 6736 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=d7546910b6e642ddb6b1345ffabe7b4a] alter complete (version 1)
I20240426 09:49:39.700750 6736 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet 8d1b192686984efcbb483742fe54850d (table rebalancer_dynamic_table [id=d7546910b6e642ddb6b1345ffabe7b4a]). Expected version 1 got 0
I20240426 09:49:39.710111 13171 tablet.cc:1722] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:39.710456 13171 tablet.cc:1620] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:39.739811 6880 consensus_queue.cc:237] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:39.741134 9672 raft_consensus.cc:1261] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:39.741190 7274 raft_consensus.cc:1261] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:39.741537 12502 consensus_queue.cc:1035] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:39.741725 13170 consensus_queue.cc:1035] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:39.743667 13170 raft_consensus.cc:2932] T 5951b29e98cc4ff3a459dec654580ec0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:39.743995 9672 raft_consensus.cc:2932] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:39.746407 7274 raft_consensus.cc:2932] T 5951b29e98cc4ff3a459dec654580ec0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:39.746629 6738 catalog_manager.cc:5484] T 5951b29e98cc4ff3a459dec654580ec0 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:39.749567 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 5951b29e98cc4ff3a459dec654580ec0 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:39.749766 12863 tablet_replica.cc:330] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:39.749904 12863 raft_consensus.cc:2229] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:39.750053 12863 raft_consensus.cc:2258] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:39.750559 12863 ts_tablet_manager.cc:1875] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:39.752761 12863 ts_tablet_manager.cc:1888] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:39.752821 12863 log.cc:1196] T 5951b29e98cc4ff3a459dec654580ec0 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/5951b29e98cc4ff3a459dec654580ec0
I20240426 09:49:39.753603 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 5951b29e98cc4ff3a459dec654580ec0 (table rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]) successfully deleted
I20240426 09:49:39.803160 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:39.810477 12435 tablet.cc:1722] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:39.810516 13171 tablet.cc:1722] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:39.810873 12435 tablet.cc:1620] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:39.810883 13171 tablet.cc:1620] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:39.810963 12429 tablet.cc:1722] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:39.811259 12429 tablet.cc:1620] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:39.816366 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=d7546910b6e642ddb6b1345ffabe7b4a] alter complete (version 2)
I20240426 09:49:39.836467 6993 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:39.844297 9397 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:39.848445 7123 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:39.851544 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:39.853842 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:39.859489 9529 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:39.862236 9265 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:39.868916 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:39.917589 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:39.924129 12429 tablet.cc:1722] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:39.924759 12435 tablet.cc:1722] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:39.925196 13171 tablet.cc:1722] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:39.931809 9549 consensus_queue.cc:237] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:39.931878 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=d7546910b6e642ddb6b1345ffabe7b4a] alter complete (version 3)
I20240426 09:49:39.933070 7011 raft_consensus.cc:1261] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:39.933436 9285 raft_consensus.cc:1261] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:39.933717 12235 consensus_queue.cc:1035] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:39.933871 12476 consensus_queue.cc:1035] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:39.935843 12235 raft_consensus.cc:2932] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:39.937423 9285 raft_consensus.cc:2932] T e3114093f8f24864b7184f0bd91fb401 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:39.940348 6734 catalog_manager.cc:5484] T e3114093f8f24864b7184f0bd91fb401 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:39.943205 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet e3114093f8f24864b7184f0bd91fb401 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:39.947117 13267 tablet_replica.cc:330] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:39.947265 13267 raft_consensus.cc:2229] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:39.947394 13267 raft_consensus.cc:2258] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:39.947768 13267 ts_tablet_manager.cc:1875] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:39.949604 13267 ts_tablet_manager.cc:1888] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:39.949676 13267 log.cc:1196] T e3114093f8f24864b7184f0bd91fb401 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/e3114093f8f24864b7184f0bd91fb401
I20240426 09:49:39.950541 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet e3114093f8f24864b7184f0bd91fb401 (table rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]) successfully deleted
I20240426 09:49:39.951072 7011 raft_consensus.cc:2932] T e3114093f8f24864b7184f0bd91fb401 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:39.999166 13232 consensus_queue.cc:1035] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.002698 13232 consensus_queue.cc:1035] T feb385fea62d4d6fa355b49461c7a05a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.029484 12920 consensus_queue.cc:1035] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.033176 12388 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.034240 12903 consensus_queue.cc:1035] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.048230 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:40.048913 6993 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:40.050419 9529 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:40.052506 6735 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:40.052644 6735 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:40.054026 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:40.054714 13119 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.055187 6735 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 8d1b192686984efcbb483742fe54850d
I20240426 09:49:40.055954 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1b192686984efcbb483742fe54850d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:40 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:40.056002 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1b192686984efcbb483742fe54850d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:40 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:40.056136 13267 tablet_replica.cc:330] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:40.056259 13267 raft_consensus.cc:2229] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:40.056450 13267 raft_consensus.cc:2258] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.057124 13267 ts_tablet_manager.cc:1875] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:40.060720 9265 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1b192686984efcbb483742fe54850d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:40 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:40.061040 13325 tablet_replica.cc:330] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:40.061298 13325 raft_consensus.cc:2229] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:40.061486 13325 raft_consensus.cc:2258] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.062003 9265 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:40.062826 13323 tablet_replica.cc:330] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:40.062930 13267 ts_tablet_manager.cc:1888] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:40.063016 13323 raft_consensus.cc:2229] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:40.063093 13267 log.cc:1196] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/8d1b192686984efcbb483742fe54850d
I20240426 09:49:40.063230 13323 raft_consensus.cc:2258] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.063409 13267 ts_tablet_manager.cc:1909] T 8d1b192686984efcbb483742fe54850d P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:40.064718 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 8d1b192686984efcbb483742fe54850d (table rebalancer_dynamic_table_new_name [id=d7546910b6e642ddb6b1345ffabe7b4a]) successfully deleted
I20240426 09:49:40.065555 13323 ts_tablet_manager.cc:1875] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:40.068008 13323 ts_tablet_manager.cc:1888] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:40.068140 13323 log.cc:1196] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/8d1b192686984efcbb483742fe54850d
I20240426 09:49:40.068583 13323 ts_tablet_manager.cc:1909] T 8d1b192686984efcbb483742fe54850d P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:40.069296 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 8d1b192686984efcbb483742fe54850d (table rebalancer_dynamic_table_new_name [id=d7546910b6e642ddb6b1345ffabe7b4a]) successfully deleted
I20240426 09:49:40.071183 13325 ts_tablet_manager.cc:1875] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:40.077760 13325 ts_tablet_manager.cc:1888] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:40.077937 13325 log.cc:1196] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/8d1b192686984efcbb483742fe54850d
I20240426 09:49:40.078226 13325 ts_tablet_manager.cc:1909] T 8d1b192686984efcbb483742fe54850d P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:40.078223 7123 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:40.079085 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 8d1b192686984efcbb483742fe54850d (table rebalancer_dynamic_table_new_name [id=d7546910b6e642ddb6b1345ffabe7b4a]) successfully deleted
I20240426 09:49:40.082283 9397 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240426 09:49:40.087234 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:40.131948 7143 consensus_queue.cc:237] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.133085 7274 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:40.133194 9548 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:40.133500 12920 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:40.133673 12903 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:40.135308 12920 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:40.135843 7274 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:40.138842 6734 catalog_manager.cc:5484] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:40.141170 9548 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:40.144887 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 0362b7d68d9048bca47d96e948563d03 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:40.145697 13349 tablet_replica.cc:330] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:40.145853 13349 raft_consensus.cc:2229] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:40.146019 13349 raft_consensus.cc:2258] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.146507 13349 ts_tablet_manager.cc:1875] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:40.147517 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:40.149044 13349 ts_tablet_manager.cc:1888] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:40.149113 13349 log.cc:1196] T 0362b7d68d9048bca47d96e948563d03 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/0362b7d68d9048bca47d96e948563d03
I20240426 09:49:40.151696 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 0362b7d68d9048bca47d96e948563d03 (table rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]) successfully deleted
I20240426 09:49:40.153507 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 2e249a2325ef4bbf9be46998f505e3fc (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.153784 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e249a2325ef4bbf9be46998f505e3fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.154456 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 2e249a2325ef4bbf9be46998f505e3fc (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.154732 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e249a2325ef4bbf9be46998f505e3fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.154718 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 2e249a2325ef4bbf9be46998f505e3fc (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.154771 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 838bef0a35c046e39b465f2ea743350f (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.155004 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e249a2325ef4bbf9be46998f505e3fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.155038 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838bef0a35c046e39b465f2ea743350f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.157245 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 91a15bb5ed1541ca906b8603d9249690 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.157361 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 838bef0a35c046e39b465f2ea743350f (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.157460 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91a15bb5ed1541ca906b8603d9249690. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.157650 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838bef0a35c046e39b465f2ea743350f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.159072 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 91a15bb5ed1541ca906b8603d9249690 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.159298 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91a15bb5ed1541ca906b8603d9249690. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.159292 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 838bef0a35c046e39b465f2ea743350f (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.159353 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 91a15bb5ed1541ca906b8603d9249690 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.159555 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838bef0a35c046e39b465f2ea743350f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.159664 13355 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:40.160009 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91a15bb5ed1541ca906b8603d9249690. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.161005 13355 tablet_bootstrap.cc:654] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.162603 13356 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:40.163082 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:40.163254 13353 tablet_bootstrap.cc:492] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:40.163539 13356 tablet_bootstrap.cc:654] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.164199 13353 tablet_bootstrap.cc:654] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.164894 13356 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:40.165016 13356 ts_tablet_manager.cc:1367] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:40.165465 13356 raft_consensus.cc:348] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.165668 13356 raft_consensus.cc:374] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.165690 13352 tablet_bootstrap.cc:492] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:40.165730 13356 raft_consensus.cc:729] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.165942 13356 consensus_queue.cc:260] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.166867 13356 ts_tablet_manager.cc:1398] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.167047 13356 tablet_bootstrap.cc:492] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:40.167897 13356 tablet_bootstrap.cc:654] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.168236 13352 tablet_bootstrap.cc:654] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.169482 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 78709e16e72a471ba5727895f901648a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3ebd24901a8041a8b917d5ea7ce5baaf]), partition=
I20240426 09:49:40.169732 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78709e16e72a471ba5727895f901648a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.169746 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 78709e16e72a471ba5727895f901648a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3ebd24901a8041a8b917d5ea7ce5baaf]), partition=
I20240426 09:49:40.169972 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78709e16e72a471ba5727895f901648a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.170150 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 78709e16e72a471ba5727895f901648a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3ebd24901a8041a8b917d5ea7ce5baaf]), partition=
I20240426 09:49:40.170362 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78709e16e72a471ba5727895f901648a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.173094 13355 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:40.173204 13355 ts_tablet_manager.cc:1367] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240426 09:49:40.173440 13353 tablet_bootstrap.cc:492] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:40.173527 13353 ts_tablet_manager.cc:1367] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:40.173758 13356 tablet_bootstrap.cc:492] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:40.173738 13355 raft_consensus.cc:348] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.173835 13355 raft_consensus.cc:374] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.173846 13356 ts_tablet_manager.cc:1367] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:40.173867 13355 raft_consensus.cc:729] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.173983 13355 consensus_queue.cc:260] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.174050 13353 raft_consensus.cc:348] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.174144 13353 raft_consensus.cc:374] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.174173 13353 raft_consensus.cc:729] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.174347 13356 raft_consensus.cc:348] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.174414 13355 ts_tablet_manager.cc:1398] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:40.174444 13356 raft_consensus.cc:374] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.174470 13356 raft_consensus.cc:729] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.174482 13353 consensus_queue.cc:260] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.174623 13355 tablet_bootstrap.cc:492] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:40.174616 13356 consensus_queue.cc:260] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.174886 13353 ts_tablet_manager.cc:1398] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:40.175576 13355 tablet_bootstrap.cc:654] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.175590 13352 tablet_bootstrap.cc:492] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:40.175679 13352 ts_tablet_manager.cc:1367] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:40.176805 13354 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:40.176162 13352 raft_consensus.cc:348] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.177502 13352 raft_consensus.cc:374] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.177542 13352 raft_consensus.cc:729] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.177680 13352 consensus_queue.cc:260] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.177753 13354 tablet_bootstrap.cc:654] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.178056 13352 ts_tablet_manager.cc:1398] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.178293 13352 tablet_bootstrap.cc:492] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:40.178970 13356 ts_tablet_manager.cc:1398] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240426 09:49:40.179383 13351 tablet_bootstrap.cc:492] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:40.180843 13351 tablet_bootstrap.cc:654] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.181221 13352 tablet_bootstrap.cc:654] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.182317 13352 tablet_bootstrap.cc:492] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:40.182320 13355 tablet_bootstrap.cc:492] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:40.182389 13352 ts_tablet_manager.cc:1367] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:40.182395 13355 ts_tablet_manager.cc:1367] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:40.182754 13352 raft_consensus.cc:348] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.182888 13352 raft_consensus.cc:374] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.182825 13355 raft_consensus.cc:348] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.182914 13352 raft_consensus.cc:729] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.182943 13355 raft_consensus.cc:374] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.182991 13355 raft_consensus.cc:729] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.183014 13352 consensus_queue.cc:260] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.183136 13355 consensus_queue.cc:260] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.183326 13352 ts_tablet_manager.cc:1398] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:40.183409 13355 ts_tablet_manager.cc:1398] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:40.183691 13355 tablet_bootstrap.cc:492] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:40.185791 13355 tablet_bootstrap.cc:654] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.186488 13362 tablet_bootstrap.cc:492] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:40.187412 13362 tablet_bootstrap.cc:654] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.188548 13362 tablet_bootstrap.cc:492] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:40.188685 13362 ts_tablet_manager.cc:1367] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:40.188724 13355 tablet_bootstrap.cc:492] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:40.188813 13355 ts_tablet_manager.cc:1367] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:40.188944 13354 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:40.189043 13354 ts_tablet_manager.cc:1367] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:49:40.189181 13362 raft_consensus.cc:348] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.189218 13355 raft_consensus.cc:348] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.189298 13355 raft_consensus.cc:374] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.189297 13362 raft_consensus.cc:374] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.189337 13362 raft_consensus.cc:729] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.189361 13355 raft_consensus.cc:729] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.189479 13354 raft_consensus.cc:348] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.189497 13362 consensus_queue.cc:260] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.189584 13354 raft_consensus.cc:374] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.189559 13355 consensus_queue.cc:260] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.189658 13354 raft_consensus.cc:729] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.189821 13354 consensus_queue.cc:260] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.190053 13354 ts_tablet_manager.cc:1398] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:40.190407 13362 ts_tablet_manager.cc:1398] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.190512 13351 tablet_bootstrap.cc:492] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:40.190608 13351 ts_tablet_manager.cc:1367] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:40.190933 13354 tablet_bootstrap.cc:492] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:40.191668 13351 raft_consensus.cc:348] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.191856 13351 raft_consensus.cc:374] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.191910 13351 raft_consensus.cc:729] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.192059 13351 consensus_queue.cc:260] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.193011 13351 ts_tablet_manager.cc:1398] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.193177 13354 tablet_bootstrap.cc:654] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.195647 13355 ts_tablet_manager.cc:1398] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:49:40.196486 13354 tablet_bootstrap.cc:492] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:40.196779 13354 ts_tablet_manager.cc:1367] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:40.197288 13354 raft_consensus.cc:348] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.197856 13354 raft_consensus.cc:374] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.198036 13354 raft_consensus.cc:729] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.198280 13354 consensus_queue.cc:260] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.199196 13354 ts_tablet_manager.cc:1398] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.213500 12931 raft_consensus.cc:482] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:40.213613 12931 raft_consensus.cc:504] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.214090 12931 leader_election.cc:288] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:40.214391 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e249a2325ef4bbf9be46998f505e3fc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:40.214529 9672 raft_consensus.cc:2454] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:40.214788 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e249a2325ef4bbf9be46998f505e3fc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:40.214845 9350 leader_election.cc:302] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:40.214927 6880 raft_consensus.cc:2454] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:40.215067 13232 raft_consensus.cc:2781] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:40.215139 13232 raft_consensus.cc:482] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:40.215175 13232 raft_consensus.cc:3037] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.216254 13232 raft_consensus.cc:504] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.216590 13232 leader_election.cc:288] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:40.216838 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e249a2325ef4bbf9be46998f505e3fc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:40.216953 9672 raft_consensus.cc:3037] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.217259 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e249a2325ef4bbf9be46998f505e3fc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:40.217353 6880 raft_consensus.cc:3037] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.217939 9672 raft_consensus.cc:2454] T 2e249a2325ef4bbf9be46998f505e3fc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:40.218189 9350 leader_election.cc:302] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:40.218345 13232 raft_consensus.cc:2781] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:40.218349 6880 raft_consensus.cc:2454] T 2e249a2325ef4bbf9be46998f505e3fc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:40.218417 13232 raft_consensus.cc:686] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:40.218544 13232 consensus_queue.cc:237] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.220485 6735 catalog_manager.cc:5484] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:40.247376 12903 raft_consensus.cc:482] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:40.247507 12903 raft_consensus.cc:504] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.247946 12903 leader_election.cc:288] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:40.248690 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78709e16e72a471ba5727895f901648a" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:40.250576 6880 raft_consensus.cc:2454] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:40.251144 7079 leader_election.cc:302] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:40.251503 12920 raft_consensus.cc:2781] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:40.251556 12920 raft_consensus.cc:482] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:40.251595 12920 raft_consensus.cc:3037] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.252722 12920 raft_consensus.cc:504] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.253064 12920 leader_election.cc:288] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:40.253451 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78709e16e72a471ba5727895f901648a" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:40.253592 7274 raft_consensus.cc:2454] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:40.254546 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78709e16e72a471ba5727895f901648a" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:40.254653 7275 raft_consensus.cc:3037] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.254684 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78709e16e72a471ba5727895f901648a" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:40.254824 6880 raft_consensus.cc:3037] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.255653 7275 raft_consensus.cc:2454] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:40.255846 6880 raft_consensus.cc:2454] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:40.256428 7079 leader_election.cc:302] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:40.256763 13344 raft_consensus.cc:2781] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:40.256840 13344 raft_consensus.cc:686] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:40.256990 13344 consensus_queue.cc:237] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.260563 6733 catalog_manager.cc:5484] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.283102 12920 raft_consensus.cc:482] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:40.283226 12920 raft_consensus.cc:504] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.283591 12920 leader_election.cc:288] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:40.283887 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91a15bb5ed1541ca906b8603d9249690" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:40.283898 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91a15bb5ed1541ca906b8603d9249690" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:40.284018 9285 raft_consensus.cc:2454] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:40.284024 9548 raft_consensus.cc:2454] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:40.284271 7078 leader_election.cc:302] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:40.284431 12920 raft_consensus.cc:2781] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:40.284483 12920 raft_consensus.cc:482] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:40.284510 12920 raft_consensus.cc:3037] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.285288 12920 raft_consensus.cc:504] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.285554 12920 leader_election.cc:288] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:40.285883 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91a15bb5ed1541ca906b8603d9249690" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:40.285995 9548 raft_consensus.cc:3037] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.286031 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91a15bb5ed1541ca906b8603d9249690" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:40.286165 9285 raft_consensus.cc:3037] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.286948 9548 raft_consensus.cc:2454] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:40.287218 9285 raft_consensus.cc:2454] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:40.287233 7077 leader_election.cc:302] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:40.287499 13344 raft_consensus.cc:2781] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:40.287580 13344 raft_consensus.cc:686] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:40.287711 13344 consensus_queue.cc:237] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.289718 6739 catalog_manager.cc:5484] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:40.291484 12903 raft_consensus.cc:482] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:40.291576 12903 raft_consensus.cc:504] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.291923 12903 leader_election.cc:288] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:40.292220 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838bef0a35c046e39b465f2ea743350f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:40.292256 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838bef0a35c046e39b465f2ea743350f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:40.292358 9548 raft_consensus.cc:2454] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:40.292376 9285 raft_consensus.cc:2454] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:40.292595 7078 leader_election.cc:302] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:40.292750 12903 raft_consensus.cc:2781] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:40.292804 12903 raft_consensus.cc:482] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:40.292842 12903 raft_consensus.cc:3037] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.293673 12903 raft_consensus.cc:504] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.293946 12903 leader_election.cc:288] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:40.294207 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838bef0a35c046e39b465f2ea743350f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:40.294317 9548 raft_consensus.cc:3037] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.294337 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838bef0a35c046e39b465f2ea743350f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:40.294443 9285 raft_consensus.cc:3037] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.295348 9548 raft_consensus.cc:2454] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:40.295349 9285 raft_consensus.cc:2454] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:40.295640 7077 leader_election.cc:302] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:40.295825 12903 raft_consensus.cc:2781] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:40.295889 12903 raft_consensus.cc:686] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:40.296015 12903 consensus_queue.cc:237] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.297693 6739 catalog_manager.cc:5484] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:40.326673 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:40.326954 9266 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:40.328706 6993 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:40.329799 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:40.338343 9397 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:40.342526 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:40.356700 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:40.369016 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:40.371054 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:40.380167 6880 raft_consensus.cc:1261] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:40.380632 7274 raft_consensus.cc:1261] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:40.380834 12903 consensus_queue.cc:1035] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.381357 12903 consensus_queue.cc:1035] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.384155 6731 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 78709e16e72a471ba5727895f901648a (table rebalancer_dynamic_table [id=3ebd24901a8041a8b917d5ea7ce5baaf]). Expected version 1 got 0
I20240426 09:49:40.384613 6739 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 78709e16e72a471ba5727895f901648a (table rebalancer_dynamic_table [id=3ebd24901a8041a8b917d5ea7ce5baaf]). Expected version 1 got 0
I20240426 09:49:40.401304 13427 tablet.cc:1722] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:40.401721 13427 tablet.cc:1620] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:40.403110 13426 tablet.cc:1722] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:40.403419 13426 tablet.cc:1620] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:40.405544 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=3ebd24901a8041a8b917d5ea7ce5baaf] alter complete (version 1)
I20240426 09:49:40.406849 13428 tablet.cc:1722] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:40.407240 13428 tablet.cc:1620] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:40.408856 6732 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 78709e16e72a471ba5727895f901648a (table rebalancer_dynamic_table [id=3ebd24901a8041a8b917d5ea7ce5baaf]). Expected version 1 got 0
I20240426 09:49:40.434172 9672 consensus_queue.cc:237] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.435429 7274 raft_consensus.cc:1261] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:40.435429 9548 raft_consensus.cc:1261] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:40.435787 12362 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:40.436211 12362 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:40.437523 12362 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.438529 7275 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.439060 9548 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.441226 6732 catalog_manager.cc:5484] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.444069 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e306113f9874277a60468d9248d45ce with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:40.444305 13267 tablet_replica.cc:330] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:40.444451 13267 raft_consensus.cc:2229] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:40.444586 13267 raft_consensus.cc:2258] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.445024 13267 ts_tablet_manager.cc:1875] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:40.446630 13267 ts_tablet_manager.cc:1888] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:40.446677 13267 log.cc:1196] T 8e306113f9874277a60468d9248d45ce P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/8e306113f9874277a60468d9248d45ce
I20240426 09:49:40.447194 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 8e306113f9874277a60468d9248d45ce (table rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]) successfully deleted
I20240426 09:49:40.511493 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:40.520545 13428 tablet.cc:1722] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:40.520867 13428 tablet.cc:1620] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:40.520879 13426 tablet.cc:1722] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:40.521219 13426 tablet.cc:1620] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:40.521193 13427 tablet.cc:1722] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:40.521459 13427 tablet.cc:1620] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:40.529757 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=3ebd24901a8041a8b917d5ea7ce5baaf] alter complete (version 2)
I20240426 09:49:40.544716 6993 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:40.557404 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:40.560388 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:40.565186 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:40.566169 9397 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:40.578161 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:40.579768 9266 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:40.593523 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:40.638260 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:40.643239 7142 consensus_queue.cc:237] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.644234 13426 tablet.cc:1722] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:40.644670 9548 raft_consensus.cc:1261] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:40.645155 12903 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:40.645733 7275 raft_consensus.cc:1261] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:40.645910 13428 tablet.cc:1722] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:40.648033 13427 tablet.cc:1722] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:40.650357 12903 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.651409 6736 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=3ebd24901a8041a8b917d5ea7ce5baaf] alter complete (version 3)
I20240426 09:49:40.651659 12903 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:40.652709 7274 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.654867 9548 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.655936 6735 catalog_manager.cc:5484] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.659183 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e010e7f54b54b04afe4e3b1df0c739f with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:40.659394 13267 tablet_replica.cc:330] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:40.659507 13267 raft_consensus.cc:2229] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:40.659595 13267 raft_consensus.cc:2258] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.659996 13267 ts_tablet_manager.cc:1875] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:40.661934 13267 ts_tablet_manager.cc:1888] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:40.661998 13267 log.cc:1196] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/5e010e7f54b54b04afe4e3b1df0c739f
I20240426 09:49:40.662568 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 5e010e7f54b54b04afe4e3b1df0c739f (table rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]) successfully deleted
I20240426 09:49:40.717772 13232 consensus_queue.cc:1035] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.719218 13344 consensus_queue.cc:1035] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.727140 12920 consensus_queue.cc:1035] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.735576 13562 consensus_queue.cc:1035] T 2e249a2325ef4bbf9be46998f505e3fc P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.757333 6739 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:40.757503 6739 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:40.759331 9397 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:40.760057 6739 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 78709e16e72a471ba5727895f901648a
I20240426 09:49:40.760913 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 78709e16e72a471ba5727895f901648a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:40 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:40.761291 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 78709e16e72a471ba5727895f901648a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:40 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:40.762070 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 78709e16e72a471ba5727895f901648a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:40 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:40.763095 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:40.764101 13583 tablet_replica.cc:330] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:40.764659 9266 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:40.766053 13584 tablet_replica.cc:330] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:40.766384 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:40.766815 13584 raft_consensus.cc:2229] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:40.766968 6993 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:40.767268 9661 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:40.767578 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:40.767578 13584 raft_consensus.cc:2258] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.783134 13584 ts_tablet_manager.cc:1875] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:40.786157 12920 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.797303 13584 ts_tablet_manager.cc:1888] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:40.797461 13584 log.cc:1196] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/78709e16e72a471ba5727895f901648a
I20240426 09:49:40.797889 13584 ts_tablet_manager.cc:1909] T 78709e16e72a471ba5727895f901648a P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:40.798756 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 78709e16e72a471ba5727895f901648a (table rebalancer_dynamic_table_new_name [id=3ebd24901a8041a8b917d5ea7ce5baaf]) successfully deleted
I20240426 09:49:40.801827 13583 raft_consensus.cc:2229] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:40.801982 13583 raft_consensus.cc:2258] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.802307 13582 tablet_replica.cc:330] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:40.802449 13582 raft_consensus.cc:2229] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:40.802564 13582 raft_consensus.cc:2258] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.802661 13583 ts_tablet_manager.cc:1875] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:40.804661 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:40.806885 13583 ts_tablet_manager.cc:1888] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:40.807523 13582 ts_tablet_manager.cc:1875] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:40.812767 13583 log.cc:1196] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/78709e16e72a471ba5727895f901648a
I20240426 09:49:40.813127 13583 ts_tablet_manager.cc:1909] T 78709e16e72a471ba5727895f901648a P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:40.813380 13582 ts_tablet_manager.cc:1888] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:40.813890 13582 log.cc:1196] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/78709e16e72a471ba5727895f901648a
I20240426 09:49:40.814285 13582 ts_tablet_manager.cc:1909] T 78709e16e72a471ba5727895f901648a P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:40.814724 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 78709e16e72a471ba5727895f901648a (table rebalancer_dynamic_table_new_name [id=3ebd24901a8041a8b917d5ea7ce5baaf]) successfully deleted
I20240426 09:49:40.815316 12903 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:40.817214 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 78709e16e72a471ba5727895f901648a (table rebalancer_dynamic_table_new_name [id=3ebd24901a8041a8b917d5ea7ce5baaf]) successfully deleted
I20240426 09:49:40.829008 6735 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:40.835582 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 536858c15b894b2f82efef38b8f51594 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.835795 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1b8658361f4c4a834e8c886909fe95 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.835888 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 536858c15b894b2f82efef38b8f51594. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.836031 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1b8658361f4c4a834e8c886909fe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.836380 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 5f5418f85c2b49ce89f82331689e1b30 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.836668 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5418f85c2b49ce89f82331689e1b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.838047 13587 tablet_bootstrap.cc:492] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:40.838410 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 5f5418f85c2b49ce89f82331689e1b30 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.838637 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5418f85c2b49ce89f82331689e1b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.838665 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 5f5418f85c2b49ce89f82331689e1b30 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.838961 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5418f85c2b49ce89f82331689e1b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.839017 13587 tablet_bootstrap.cc:654] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.839577 13588 tablet_bootstrap.cc:492] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:40.840436 13588 tablet_bootstrap.cc:654] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.840704 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 536858c15b894b2f82efef38b8f51594 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.840972 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 536858c15b894b2f82efef38b8f51594. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.841399 13587 tablet_bootstrap.cc:492] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:40.841475 13588 tablet_bootstrap.cc:492] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:40.841475 13587 ts_tablet_manager.cc:1367] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:40.841540 13588 ts_tablet_manager.cc:1367] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.841854 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 536858c15b894b2f82efef38b8f51594 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.841931 13587 raft_consensus.cc:348] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:40.842133 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 536858c15b894b2f82efef38b8f51594. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.841935 13588 raft_consensus.cc:348] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.842206 13588 raft_consensus.cc:374] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.842245 13588 raft_consensus.cc:729] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.842036 13587 raft_consensus.cc:374] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.842386 13587 raft_consensus.cc:729] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.842378 13588 consensus_queue.cc:260] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.842723 13588 ts_tablet_manager.cc:1398] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:40.842670 13587 consensus_queue.cc:260] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:40.843050 13587 ts_tablet_manager.cc:1398] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.843540 13587 tablet_bootstrap.cc:492] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:40.844501 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1b8658361f4c4a834e8c886909fe95 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.844723 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1b8658361f4c4a834e8c886909fe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.844817 13587 tablet_bootstrap.cc:654] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.846709 13593 tablet_bootstrap.cc:492] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:40.847625 13593 tablet_bootstrap.cc:654] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.847836 13587 tablet_bootstrap.cc:492] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:40.847915 13587 ts_tablet_manager.cc:1367] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:40.848315 13587 raft_consensus.cc:348] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.848454 13587 raft_consensus.cc:374] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.848502 13587 raft_consensus.cc:729] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.848722 13587 consensus_queue.cc:260] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.849350 13589 tablet_bootstrap.cc:492] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:40.849785 13587 ts_tablet_manager.cc:1398] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.850421 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1b8658361f4c4a834e8c886909fe95 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:40.850690 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1b8658361f4c4a834e8c886909fe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.850780 13589 tablet_bootstrap.cc:654] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.851920 13589 tablet_bootstrap.cc:492] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:40.852016 13589 ts_tablet_manager.cc:1367] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:40.852448 13589 raft_consensus.cc:348] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.852550 13589 raft_consensus.cc:374] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.852588 13589 raft_consensus.cc:729] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.852712 13589 consensus_queue.cc:260] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.852875 13587 tablet_bootstrap.cc:492] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:40.853376 13592 tablet_bootstrap.cc:492] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:40.853778 13589 ts_tablet_manager.cc:1398] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.853871 13587 tablet_bootstrap.cc:654] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.854315 13592 tablet_bootstrap.cc:654] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.855486 13593 tablet_bootstrap.cc:492] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:40.855614 13593 ts_tablet_manager.cc:1367] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:40.855813 13587 tablet_bootstrap.cc:492] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:40.856089 13587 ts_tablet_manager.cc:1367] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:40.856104 13593 raft_consensus.cc:348] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:40.856194 13593 raft_consensus.cc:374] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.856215 13593 raft_consensus.cc:729] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.856315 13593 consensus_queue.cc:260] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:40.856577 13593 ts_tablet_manager.cc:1398] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:40.856626 13587 raft_consensus.cc:348] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.856758 13587 raft_consensus.cc:374] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.856814 13593 tablet_bootstrap.cc:492] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:40.856817 13587 raft_consensus.cc:729] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.856935 13587 consensus_queue.cc:260] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.857569 13587 ts_tablet_manager.cc:1398] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:40.857844 13593 tablet_bootstrap.cc:654] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.859092 13592 tablet_bootstrap.cc:492] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:40.860329 13592 ts_tablet_manager.cc:1367] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:40.860877 13592 raft_consensus.cc:348] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.861369 13593 tablet_bootstrap.cc:492] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:40.861407 13592 raft_consensus.cc:374] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.861496 13593 ts_tablet_manager.cc:1367] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:40.861515 13592 raft_consensus.cc:729] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.861726 13592 consensus_queue.cc:260] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:40.861970 13593 raft_consensus.cc:348] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.862076 13593 raft_consensus.cc:374] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.862115 13593 raft_consensus.cc:729] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.862051 13592 ts_tablet_manager.cc:1398] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.862140 6735 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:40.862274 13593 consensus_queue.cc:260] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:40.862588 13593 ts_tablet_manager.cc:1398] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:40.863260 13592 tablet_bootstrap.cc:492] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:40.864892 13592 tablet_bootstrap.cc:654] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.867467 13592 tablet_bootstrap.cc:492] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:40.867784 13592 ts_tablet_manager.cc:1367] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:40.868270 13592 raft_consensus.cc:348] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:40.868422 13592 raft_consensus.cc:374] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.868484 13592 raft_consensus.cc:729] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.868625 13592 consensus_queue.cc:260] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:40.868927 13592 ts_tablet_manager.cc:1398] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:40.870975 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 51ca8d641b3748929140bb882423ca7b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=225835d8a00c433a9c059fe8ac5dc387]), partition=
I20240426 09:49:40.871284 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51ca8d641b3748929140bb882423ca7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.871328 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 51ca8d641b3748929140bb882423ca7b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=225835d8a00c433a9c059fe8ac5dc387]), partition=
I20240426 09:49:40.871490 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 51ca8d641b3748929140bb882423ca7b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=225835d8a00c433a9c059fe8ac5dc387]), partition=
I20240426 09:49:40.871555 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51ca8d641b3748929140bb882423ca7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.871698 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51ca8d641b3748929140bb882423ca7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:40.873327 13587 tablet_bootstrap.cc:492] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:40.875072 13612 tablet_bootstrap.cc:492] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:40.875411 13613 tablet_bootstrap.cc:492] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:40.876108 13587 tablet_bootstrap.cc:654] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.876255 13612 tablet_bootstrap.cc:654] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.876809 13613 tablet_bootstrap.cc:654] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:40.877537 13587 tablet_bootstrap.cc:492] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:40.877671 13587 ts_tablet_manager.cc:1367] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:40.877894 13612 tablet_bootstrap.cc:492] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:40.877995 13612 ts_tablet_manager.cc:1367] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:40.878387 13587 raft_consensus.cc:348] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.878487 13587 raft_consensus.cc:374] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.878522 13587 raft_consensus.cc:729] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.878500 13612 raft_consensus.cc:348] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.878679 13612 raft_consensus.cc:374] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.878669 13587 consensus_queue.cc:260] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.878962 13587 ts_tablet_manager.cc:1398] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:40.878818 13612 raft_consensus.cc:729] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.879328 13612 consensus_queue.cc:260] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.879659 13612 ts_tablet_manager.cc:1398] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:40.880999 13613 tablet_bootstrap.cc:492] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:40.881099 13613 ts_tablet_manager.cc:1367] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:40.882093 13613 raft_consensus.cc:348] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.882848 13613 raft_consensus.cc:374] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:40.883000 13613 raft_consensus.cc:729] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:40.883891 13613 consensus_queue.cc:260] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:40.884228 13613 ts_tablet_manager.cc:1398] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:40.887691 9672 consensus_queue.cc:237] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:40.888936 7274 raft_consensus.cc:1261] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:40.888980 6882 raft_consensus.cc:1261] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:40.889355 13447 consensus_queue.cc:1035] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:40.889534 13321 consensus_queue.cc:1035] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:40.891196 12362 raft_consensus.cc:2932] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.892884 7274 raft_consensus.cc:2932] T ce43c4c9b2994813ad7ad38f2e1b1040 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.893193 6882 raft_consensus.cc:2932] T ce43c4c9b2994813ad7ad38f2e1b1040 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.895143 6735 catalog_manager.cc:5484] T ce43c4c9b2994813ad7ad38f2e1b1040 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:40.898079 9265 tablet_service.cc:1514] Processing DeleteTablet for tablet ce43c4c9b2994813ad7ad38f2e1b1040 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:40.898749 13629 tablet_replica.cc:330] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:40.898972 13629 raft_consensus.cc:2229] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:40.899168 13629 raft_consensus.cc:2258] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:40.899662 13629 ts_tablet_manager.cc:1875] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:40.901880 13629 ts_tablet_manager.cc:1888] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:40.901965 13629 log.cc:1196] T ce43c4c9b2994813ad7ad38f2e1b1040 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/ce43c4c9b2994813ad7ad38f2e1b1040
I20240426 09:49:40.902535 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet ce43c4c9b2994813ad7ad38f2e1b1040 (table rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]) successfully deleted
I20240426 09:49:40.902565 13569 raft_consensus.cc:482] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:40.902667 13569 raft_consensus.cc:504] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:40.903133 13569 leader_election.cc:288] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:40.903570 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "536858c15b894b2f82efef38b8f51594" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:40.903575 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "536858c15b894b2f82efef38b8f51594" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:40.903685 7013 raft_consensus.cc:2454] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:40.903685 9285 raft_consensus.cc:2454] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:40.903955 9351 leader_election.cc:302] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:40.904153 13569 raft_consensus.cc:2781] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:40.904217 13569 raft_consensus.cc:482] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:40.904260 13569 raft_consensus.cc:3037] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.905261 13569 raft_consensus.cc:504] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:40.905593 13569 leader_election.cc:288] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:40.905874 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "536858c15b894b2f82efef38b8f51594" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:40.905992 7013 raft_consensus.cc:3037] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.906373 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "536858c15b894b2f82efef38b8f51594" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:40.906472 9285 raft_consensus.cc:3037] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:40.907009 7013 raft_consensus.cc:2454] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:40.907414 9285 raft_consensus.cc:2454] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:40.907714 9350 leader_election.cc:302] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:40.907866 13569 raft_consensus.cc:2781] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:40.907955 13569 raft_consensus.cc:686] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:40.908087 13569 consensus_queue.cc:237] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:40.909955 6739 catalog_manager.cc:5484] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:41.037043 13668 raft_consensus.cc:482] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:41.037160 13668 raft_consensus.cc:504] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.037549 13668 leader_election.cc:288] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:41.037798 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51ca8d641b3748929140bb882423ca7b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:41.037952 9416 raft_consensus.cc:2454] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:41.037927 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51ca8d641b3748929140bb882423ca7b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:41.038082 6882 raft_consensus.cc:2454] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:41.038231 7208 leader_election.cc:302] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:41.038494 13668 raft_consensus.cc:2781] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:41.038555 13668 raft_consensus.cc:482] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:41.038591 13668 raft_consensus.cc:3037] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.039621 13668 raft_consensus.cc:504] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.039959 13668 leader_election.cc:288] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:41.040261 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51ca8d641b3748929140bb882423ca7b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:41.040268 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51ca8d641b3748929140bb882423ca7b" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:41.040374 9416 raft_consensus.cc:3037] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.040369 6882 raft_consensus.cc:3037] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.041074 9416 raft_consensus.cc:2454] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:41.041111 6882 raft_consensus.cc:2454] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:41.041428 7210 leader_election.cc:302] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:41.041591 13669 raft_consensus.cc:2781] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:41.041718 13669 raft_consensus.cc:686] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:41.041915 13669 consensus_queue.cc:237] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.043869 6739 catalog_manager.cc:5484] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:41.130396 13601 raft_consensus.cc:482] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:41.130514 13601 raft_consensus.cc:504] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:41.130951 13601 leader_election.cc:288] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:41.131356 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5418f85c2b49ce89f82331689e1b30" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:41.131384 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5418f85c2b49ce89f82331689e1b30" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:41.131505 9416 raft_consensus.cc:2454] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:41.131505 9548 raft_consensus.cc:2454] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:41.131773 6946 leader_election.cc:302] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:41.131929 13601 raft_consensus.cc:2781] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:41.132000 13601 raft_consensus.cc:482] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:41.132057 13601 raft_consensus.cc:3037] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.133060 13601 raft_consensus.cc:504] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:41.133364 13601 leader_election.cc:288] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:41.133674 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5418f85c2b49ce89f82331689e1b30" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:41.133702 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5418f85c2b49ce89f82331689e1b30" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:41.133785 9416 raft_consensus.cc:3037] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.133803 9548 raft_consensus.cc:3037] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.134621 9548 raft_consensus.cc:2454] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:41.134737 9416 raft_consensus.cc:2454] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:41.134883 6946 leader_election.cc:302] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:41.135066 13601 raft_consensus.cc:2781] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:41.135151 13601 raft_consensus.cc:686] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:41.135288 13601 consensus_queue.cc:237] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:41.137131 6739 catalog_manager.cc:5484] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:41.164973 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:41.166343 13321 raft_consensus.cc:482] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:41.166466 13321 raft_consensus.cc:504] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.166886 13321 leader_election.cc:288] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:41.167308 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1b8658361f4c4a834e8c886909fe95" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:41.167404 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1b8658361f4c4a834e8c886909fe95" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:41.167459 9285 raft_consensus.cc:2454] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:41.167526 9416 raft_consensus.cc:2454] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:41.167785 9614 leader_election.cc:302] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:41.167948 13321 raft_consensus.cc:2781] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:41.168015 13321 raft_consensus.cc:482] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:41.168074 13321 raft_consensus.cc:3037] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.169198 13321 raft_consensus.cc:504] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.169515 13321 leader_election.cc:288] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:41.169806 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1b8658361f4c4a834e8c886909fe95" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:41.169833 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1b8658361f4c4a834e8c886909fe95" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:41.169916 9416 raft_consensus.cc:3037] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.169939 9285 raft_consensus.cc:3037] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.170876 9416 raft_consensus.cc:2454] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:41.171113 9285 raft_consensus.cc:2454] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:41.171319 9614 leader_election.cc:302] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:41.171471 13321 raft_consensus.cc:2781] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:41.171530 13321 raft_consensus.cc:686] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:41.171641 13321 consensus_queue.cc:237] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.171890 9416 raft_consensus.cc:1261] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:41.172243 13668 consensus_queue.cc:1035] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:41.172580 6882 raft_consensus.cc:1261] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:41.173229 13669 consensus_queue.cc:1035] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:41.175093 6731 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 51ca8d641b3748929140bb882423ca7b (table rebalancer_dynamic_table [id=225835d8a00c433a9c059fe8ac5dc387]). Expected version 1 got 0
I20240426 09:49:41.175702 6739 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 51ca8d641b3748929140bb882423ca7b (table rebalancer_dynamic_table [id=225835d8a00c433a9c059fe8ac5dc387]). Expected version 1 got 0
I20240426 09:49:41.175936 6736 catalog_manager.cc:5484] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:41.176465 13677 tablet.cc:1722] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:41.176822 13677 tablet.cc:1620] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:41.179209 13678 tablet.cc:1722] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:41.179415 6736 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 51ca8d641b3748929140bb882423ca7b (table rebalancer_dynamic_table [id=225835d8a00c433a9c059fe8ac5dc387]). Expected version 1 got 0
I20240426 09:49:41.179656 13678 tablet.cc:1620] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:41.179742 13676 tablet.cc:1722] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:41.180400 13676 tablet.cc:1620] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:41.183256 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=225835d8a00c433a9c059fe8ac5dc387] alter complete (version 1)
I20240426 09:49:41.212530 6993 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:41.214138 9265 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:41.215313 9397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:41.216387 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:41.216877 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:41.217124 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:41.232800 9661 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:41.251117 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:41.283901 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:41.288730 13676 tablet.cc:1722] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:41.289059 13676 tablet.cc:1620] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:41.289855 13677 tablet.cc:1722] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:41.290136 13677 tablet.cc:1620] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:41.291440 13678 tablet.cc:1722] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:41.291718 13678 tablet.cc:1620] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:41.295917 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=225835d8a00c433a9c059fe8ac5dc387] alter complete (version 2)
I20240426 09:49:41.296413 13562 consensus_queue.cc:1035] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:41.299952 13562 consensus_queue.cc:1035] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:41.305428 7274 consensus_queue.cc:237] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.306594 9548 raft_consensus.cc:1261] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:41.306605 6882 raft_consensus.cc:1261] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:41.306985 13668 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:41.307168 12838 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:41.308759 13669 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:41.309201 6882 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:41.311395 9548 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:41.312323 6739 catalog_manager.cc:5484] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:41.315765 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 722306427b7b46b4b0942193c821d990 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:41.316100 13716 tablet_replica.cc:330] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:41.316257 13716 raft_consensus.cc:2229] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:41.316386 13716 raft_consensus.cc:2258] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:41.316816 13716 ts_tablet_manager.cc:1875] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:41.318953 13716 ts_tablet_manager.cc:1888] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:41.319038 13716 log.cc:1196] T 722306427b7b46b4b0942193c821d990 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/722306427b7b46b4b0942193c821d990
I20240426 09:49:41.319612 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 722306427b7b46b4b0942193c821d990 (table rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]) successfully deleted
I20240426 09:49:41.399542 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:41.406055 13676 tablet.cc:1722] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:41.409169 13677 tablet.cc:1722] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:41.414459 13678 tablet.cc:1722] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:41.414717 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=225835d8a00c433a9c059fe8ac5dc387] alter complete (version 3)
I20240426 09:49:41.423048 6993 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:41.423862 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:41.429875 9397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:41.432183 9265 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240426 09:49:41.442917 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:41.443146 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:41.450521 9661 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:41.459307 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:41.508572 7013 consensus_queue.cc:237] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:41.509912 9548 raft_consensus.cc:1261] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:49:41.509925 7142 raft_consensus.cc:1261] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:49:41.510717 13601 consensus_queue.cc:1035] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:49:41.510903 12905 consensus_queue.cc:1035] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:49:41.512832 12905 raft_consensus.cc:2932] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:41.513239 9548 raft_consensus.cc:2932] T b0a674c2acbe4f69a7b8fd30fa12b833 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:41.515849 6735 catalog_manager.cc:5484] T b0a674c2acbe4f69a7b8fd30fa12b833 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 4 to 6, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:41.516695 6731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:41.516813 6731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:41.519450 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 51ca8d641b3748929140bb882423ca7b
I20240426 09:49:41.519476 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a674c2acbe4f69a7b8fd30fa12b833 with delete_type TABLET_DATA_TOMBSTONED (TS c244e02f3cf847c0a5b69e1059f1071c not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:41.519857 7142 raft_consensus.cc:2932] T b0a674c2acbe4f69a7b8fd30fa12b833 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:41.520474 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 51ca8d641b3748929140bb882423ca7b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:41 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:41.520534 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 51ca8d641b3748929140bb882423ca7b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:41 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:41.521343 13798 tablet_replica.cc:330] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:41.521584 13798 raft_consensus.cc:2229] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:41.521924 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 51ca8d641b3748929140bb882423ca7b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:41 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:41.522255 13798 raft_consensus.cc:2258] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:41.522975 13795 tablet_replica.cc:330] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:41.523161 13795 raft_consensus.cc:2229] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:41.523520 13797 tablet_replica.cc:330] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:41.523737 13797 raft_consensus.cc:2229] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:41.523313 13795 raft_consensus.cc:2258] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:41.523967 13797 raft_consensus.cc:2258] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:41.524392 13795 ts_tablet_manager.cc:1875] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:41.525007 13798 ts_tablet_manager.cc:1875] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:41.526329 13797 ts_tablet_manager.cc:1875] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:41.526965 13795 ts_tablet_manager.cc:1888] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:41.527071 13795 log.cc:1196] T b0a674c2acbe4f69a7b8fd30fa12b833 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/b0a674c2acbe4f69a7b8fd30fa12b833
I20240426 09:49:41.527559 13795 tablet_replica.cc:330] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:41.527670 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet b0a674c2acbe4f69a7b8fd30fa12b833 (table rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]) successfully deleted
I20240426 09:49:41.527716 13795 raft_consensus.cc:2229] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:41.527856 13795 raft_consensus.cc:2258] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:41.528481 13798 ts_tablet_manager.cc:1888] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:41.528612 13795 ts_tablet_manager.cc:1875] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:41.528847 13798 log.cc:1196] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/51ca8d641b3748929140bb882423ca7b
I20240426 09:49:41.529217 13798 ts_tablet_manager.cc:1909] T 51ca8d641b3748929140bb882423ca7b P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:41.531713 13797 ts_tablet_manager.cc:1888] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:41.531847 13797 log.cc:1196] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/51ca8d641b3748929140bb882423ca7b
I20240426 09:49:41.531842 13795 ts_tablet_manager.cc:1888] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:41.531903 13795 log.cc:1196] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/51ca8d641b3748929140bb882423ca7b
I20240426 09:49:41.532204 13795 ts_tablet_manager.cc:1909] T 51ca8d641b3748929140bb882423ca7b P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:41.532204 13797 ts_tablet_manager.cc:1909] T 51ca8d641b3748929140bb882423ca7b P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:41.532618 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 51ca8d641b3748929140bb882423ca7b (table rebalancer_dynamic_table_new_name [id=225835d8a00c433a9c059fe8ac5dc387]) successfully deleted
I20240426 09:49:41.532763 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 51ca8d641b3748929140bb882423ca7b (table rebalancer_dynamic_table_new_name [id=225835d8a00c433a9c059fe8ac5dc387]) successfully deleted
I20240426 09:49:41.532829 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 51ca8d641b3748929140bb882423ca7b (table rebalancer_dynamic_table_new_name [id=225835d8a00c433a9c059fe8ac5dc387]) successfully deleted
I20240426 09:49:41.559437 6882 consensus_queue.cc:237] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.560734 13789 consensus_queue.cc:1035] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240426 09:49:41.560981 6816 consensus_peers.cc:485] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 948e1df8d52f42cebcd289fff377d6fb. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:41.561784 7142 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:41.562319 7274 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:41.562819 13770 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:41.563185 13770 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:41.564913 13601 consensus_queue.cc:1035] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:41.571630 13770 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:41.572114 7142 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:41.574121 7274 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:41.574808 6739 catalog_manager.cc:5484] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:41.595561 7274 consensus_queue.cc:237] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.596947 6882 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:41.597095 7013 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:41.597419 13668 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:49:41.597600 7208 consensus_peers.cc:485] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4189112964fb425aac074352dd873415. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:41.597795 13668 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:41.599395 13668 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:41.599849 7013 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:41.601552 6882 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:41.603271 6731 catalog_manager.cc:5484] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:41.622263 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:41.627704 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 24146d69f1a147e7933836f4b44b41bf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=05080a15410a41799c116600c206843d]), partition=
I20240426 09:49:41.628015 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24146d69f1a147e7933836f4b44b41bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.628026 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 24146d69f1a147e7933836f4b44b41bf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=05080a15410a41799c116600c206843d]), partition=
I20240426 09:49:41.628227 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24146d69f1a147e7933836f4b44b41bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.630265 13851 tablet_bootstrap.cc:492] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:41.630445 13852 tablet_bootstrap.cc:492] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:41.630718 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 24146d69f1a147e7933836f4b44b41bf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=05080a15410a41799c116600c206843d]), partition=
I20240426 09:49:41.630998 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24146d69f1a147e7933836f4b44b41bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.631479 13852 tablet_bootstrap.cc:654] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.631479 13851 tablet_bootstrap.cc:654] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.632834 13851 tablet_bootstrap.cc:492] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:41.632899 13851 ts_tablet_manager.cc:1367] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:41.633175 13855 tablet_bootstrap.cc:492] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:41.633359 13851 raft_consensus.cc:348] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.633507 13851 raft_consensus.cc:374] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.633561 13851 raft_consensus.cc:729] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.633738 13851 consensus_queue.cc:260] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.634150 13851 ts_tablet_manager.cc:1398] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:41.634884 13855 tablet_bootstrap.cc:654] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.635250 13852 tablet_bootstrap.cc:492] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:41.635344 13852 ts_tablet_manager.cc:1367] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:41.635828 13852 raft_consensus.cc:348] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.635926 13852 raft_consensus.cc:374] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.635954 13852 raft_consensus.cc:729] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.636075 13852 consensus_queue.cc:260] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.636332 13852 ts_tablet_manager.cc:1398] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:41.637959 13855 tablet_bootstrap.cc:492] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:41.638051 13855 ts_tablet_manager.cc:1367] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240426 09:49:41.638499 13855 raft_consensus.cc:348] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.638577 13855 raft_consensus.cc:374] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.638597 13855 raft_consensus.cc:729] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.638697 13855 consensus_queue.cc:260] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.639005 13855 ts_tablet_manager.cc:1398] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:41.641119 12362 consensus_queue.cc:1035] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:41.645654 12362 consensus_queue.cc:1035] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:41.686131 12388 raft_consensus.cc:482] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:41.686281 12388 raft_consensus.cc:504] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.686728 12388 leader_election.cc:288] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:41.687218 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24146d69f1a147e7933836f4b44b41bf" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:41.687218 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24146d69f1a147e7933836f4b44b41bf" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:41.687350 9548 raft_consensus.cc:2454] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:41.687355 9672 raft_consensus.cc:2454] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:41.687669 9219 leader_election.cc:302] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:41.687928 12388 raft_consensus.cc:2781] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:41.687981 12388 raft_consensus.cc:482] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:41.688016 12388 raft_consensus.cc:3037] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.689111 12388 raft_consensus.cc:504] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.689500 12388 leader_election.cc:288] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:41.689872 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24146d69f1a147e7933836f4b44b41bf" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:41.689890 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24146d69f1a147e7933836f4b44b41bf" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:41.690001 9548 raft_consensus.cc:3037] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.690016 9672 raft_consensus.cc:3037] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.691063 9548 raft_consensus.cc:2454] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:41.691061 9672 raft_consensus.cc:2454] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:41.691372 9219 leader_election.cc:302] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:41.691784 12388 raft_consensus.cc:2781] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:41.691874 12388 raft_consensus.cc:686] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:41.692009 12388 consensus_queue.cc:237] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.694653 6731 catalog_manager.cc:5484] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:41.722862 6993 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:41.725787 9661 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:41.728565 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:41.730213 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:41.738889 9265 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:41.739974 9660 tablet_service.cc:1467] Processing CreateTablet for tablet da33173a2b0c45ba95cebdf61c6a7499 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:41.740258 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da33173a2b0c45ba95cebdf61c6a7499. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.740332 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 6112d2390c994fefb74b8e7f75d57d17 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:41.740613 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6112d2390c994fefb74b8e7f75d57d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.742549 13851 tablet_bootstrap.cc:492] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:41.742898 13855 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:41.743479 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 6112d2390c994fefb74b8e7f75d57d17 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:41.743691 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6112d2390c994fefb74b8e7f75d57d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.744277 13855 tablet_bootstrap.cc:654] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.745055 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 1a99c72632ab49a5b7b2c4a5b305b119 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:41.745301 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a99c72632ab49a5b7b2c4a5b305b119. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.746279 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 6112d2390c994fefb74b8e7f75d57d17 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:41.746342 9396 tablet_service.cc:1467] Processing CreateTablet for tablet da33173a2b0c45ba95cebdf61c6a7499 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:41.746523 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6112d2390c994fefb74b8e7f75d57d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.746650 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da33173a2b0c45ba95cebdf61c6a7499. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.747169 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 1a99c72632ab49a5b7b2c4a5b305b119 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:41.747462 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a99c72632ab49a5b7b2c4a5b305b119. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.742915 9266 tablet_service.cc:1467] Processing CreateTablet for tablet da33173a2b0c45ba95cebdf61c6a7499 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:41.747982 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da33173a2b0c45ba95cebdf61c6a7499. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.748912 13903 tablet_bootstrap.cc:492] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:41.749888 13903 tablet_bootstrap.cc:654] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.751087 13903 tablet_bootstrap.cc:492] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:41.751174 13903 ts_tablet_manager.cc:1367] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:41.751673 13903 raft_consensus.cc:348] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.751775 13903 raft_consensus.cc:374] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.751801 13903 raft_consensus.cc:729] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.751910 13903 consensus_queue.cc:260] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.752174 13903 ts_tablet_manager.cc:1398] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:41.752818 13851 tablet_bootstrap.cc:654] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.755652 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 1a99c72632ab49a5b7b2c4a5b305b119 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:41.755983 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a99c72632ab49a5b7b2c4a5b305b119. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:41.756918 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:41.757184 13855 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:41.757275 13855 ts_tablet_manager.cc:1367] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20240426 09:49:41.757701 13855 raft_consensus.cc:348] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.757810 13855 raft_consensus.cc:374] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.757838 13855 raft_consensus.cc:729] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.757952 13855 consensus_queue.cc:260] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.758210 13855 ts_tablet_manager.cc:1398] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:41.758361 13855 tablet_bootstrap.cc:492] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:41.758374 9397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:41.759881 13851 tablet_bootstrap.cc:492] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:41.759960 13851 ts_tablet_manager.cc:1367] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240426 09:49:41.760443 13851 raft_consensus.cc:348] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.760566 13851 raft_consensus.cc:374] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.760604 13851 raft_consensus.cc:729] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.760749 13851 consensus_queue.cc:260] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.761041 13851 ts_tablet_manager.cc:1398] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:41.761247 13851 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:41.762241 13851 tablet_bootstrap.cc:654] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.765188 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:41.765594 7255 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:41.770884 13855 tablet_bootstrap.cc:654] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.773723 13905 tablet_bootstrap.cc:492] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:41.777705 13905 tablet_bootstrap.cc:654] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.779911 13851 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:41.780026 13851 ts_tablet_manager.cc:1367] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.001s
I20240426 09:49:41.780526 13851 raft_consensus.cc:348] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.780647 13851 raft_consensus.cc:374] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.780670 13851 raft_consensus.cc:729] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.780778 13851 consensus_queue.cc:260] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.781069 13851 ts_tablet_manager.cc:1398] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:41.781303 13851 tablet_bootstrap.cc:492] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:41.782284 13851 tablet_bootstrap.cc:654] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.799201 6734 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:41.807978 9672 raft_consensus.cc:1261] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:41.808415 9548 raft_consensus.cc:1261] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:41.808895 13787 consensus_queue.cc:1035] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:41.809170 13787 consensus_queue.cc:1035] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:41.810868 13904 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:41.811676 13851 tablet_bootstrap.cc:492] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:41.811769 13851 ts_tablet_manager.cc:1367] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.002s
I20240426 09:49:41.812219 13904 tablet_bootstrap.cc:654] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.812345 13851 raft_consensus.cc:348] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.812471 13851 raft_consensus.cc:374] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.812500 13851 raft_consensus.cc:729] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.812614 13851 consensus_queue.cc:260] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.812904 13851 ts_tablet_manager.cc:1398] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:41.813316 13855 tablet_bootstrap.cc:492] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:41.813441 13855 ts_tablet_manager.cc:1367] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.055s user 0.000s sys 0.002s
I20240426 09:49:41.813907 13855 raft_consensus.cc:348] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.814018 13855 raft_consensus.cc:374] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.814055 13855 raft_consensus.cc:729] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.814170 13855 consensus_queue.cc:260] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.814441 13855 ts_tablet_manager.cc:1398] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:41.815014 6734 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet 24146d69f1a147e7933836f4b44b41bf (table rebalancer_dynamic_table [id=05080a15410a41799c116600c206843d]). Expected version 1 got 0
I20240426 09:49:41.816206 6731 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet 24146d69f1a147e7933836f4b44b41bf (table rebalancer_dynamic_table [id=05080a15410a41799c116600c206843d]). Expected version 1 got 0
I20240426 09:49:41.820161 13904 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:41.820344 13904 ts_tablet_manager.cc:1367] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:41.821045 13904 raft_consensus.cc:348] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.821254 13904 raft_consensus.cc:374] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.821331 13904 raft_consensus.cc:729] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.821384 13905 tablet_bootstrap.cc:492] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:41.821491 13905 ts_tablet_manager.cc:1367] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.048s user 0.003s sys 0.000s
I20240426 09:49:41.821517 13904 consensus_queue.cc:260] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:41.821820 13904 ts_tablet_manager.cc:1398] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:41.821990 13904 tablet_bootstrap.cc:492] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:41.821941 13905 raft_consensus.cc:348] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.822052 13905 raft_consensus.cc:374] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.822082 13905 raft_consensus.cc:729] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.822217 13905 consensus_queue.cc:260] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.822471 13905 ts_tablet_manager.cc:1398] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:41.824316 13904 tablet_bootstrap.cc:654] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:41.824975 12838 raft_consensus.cc:482] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:41.825093 12838 raft_consensus.cc:504] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.825513 12838 leader_election.cc:288] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:41.826324 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a99c72632ab49a5b7b2c4a5b305b119" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:41.826392 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a99c72632ab49a5b7b2c4a5b305b119" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:41.826437 6882 raft_consensus.cc:2454] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:41.826498 9672 raft_consensus.cc:2454] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:41.827450 7208 leader_election.cc:302] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:41.828166 13919 tablet.cc:1722] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:41.828832 6731 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet 24146d69f1a147e7933836f4b44b41bf (table rebalancer_dynamic_table [id=05080a15410a41799c116600c206843d]). Expected version 1 got 0
I20240426 09:49:41.829247 6736 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet 24146d69f1a147e7933836f4b44b41bf (table rebalancer_dynamic_table [id=05080a15410a41799c116600c206843d]). Expected version 1 got 0
I20240426 09:49:41.829806 13919 tablet.cc:1620] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:41.830160 12838 raft_consensus.cc:2781] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:41.830230 12838 raft_consensus.cc:482] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:41.830263 12838 raft_consensus.cc:3037] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.830493 13918 tablet.cc:1722] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:41.830822 13918 tablet.cc:1620] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:41.831300 12838 raft_consensus.cc:504] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.831679 12838 leader_election.cc:288] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:41.832170 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a99c72632ab49a5b7b2c4a5b305b119" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:41.832283 6882 raft_consensus.cc:3037] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.832257 13917 tablet.cc:1722] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:41.832494 13917 tablet.cc:1620] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:41.833252 6882 raft_consensus.cc:2454] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:41.833588 7210 leader_election.cc:302] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:41.834127 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a99c72632ab49a5b7b2c4a5b305b119" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:41.834234 13904 tablet_bootstrap.cc:492] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:41.834242 9672 raft_consensus.cc:3037] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.834319 13904 ts_tablet_manager.cc:1367] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20240426 09:49:41.835227 9672 raft_consensus.cc:2454] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:41.836336 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=05080a15410a41799c116600c206843d] alter complete (version 1)
I20240426 09:49:41.836789 13904 raft_consensus.cc:348] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.837738 13904 raft_consensus.cc:374] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:41.837788 13904 raft_consensus.cc:729] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:41.837924 13904 consensus_queue.cc:260] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.838196 13904 ts_tablet_manager.cc:1398] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240426 09:49:41.839866 12838 raft_consensus.cc:2781] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:41.839991 12838 raft_consensus.cc:686] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:41.840127 12838 consensus_queue.cc:237] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:41.840452 13916 raft_consensus.cc:482] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:41.840551 13916 raft_consensus.cc:504] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.841145 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da33173a2b0c45ba95cebdf61c6a7499" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:41.841279 9416 raft_consensus.cc:2454] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:41.841626 9219 leader_election.cc:302] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:41.841778 13787 raft_consensus.cc:2781] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:41.841841 13787 raft_consensus.cc:482] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:41.841876 13787 raft_consensus.cc:3037] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.842306 6739 catalog_manager.cc:5484] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:41.843114 13787 raft_consensus.cc:504] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.843477 13787 leader_election.cc:288] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:41.843828 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da33173a2b0c45ba95cebdf61c6a7499" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:41.843889 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da33173a2b0c45ba95cebdf61c6a7499" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:41.843950 9672 raft_consensus.cc:3037] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.843991 9416 raft_consensus.cc:3037] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:41.844838 9416 raft_consensus.cc:2454] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:41.844887 9672 raft_consensus.cc:2454] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:41.845876 13916 leader_election.cc:288] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:41.846022 9219 leader_election.cc:302] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:41.846212 13916 raft_consensus.cc:2781] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:41.846350 13916 raft_consensus.cc:686] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:41.846438 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da33173a2b0c45ba95cebdf61c6a7499" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:41.846560 9672 raft_consensus.cc:2362] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1. Re-sending same reply.
I20240426 09:49:41.846571 13916 consensus_queue.cc:237] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:41.848501 6739 catalog_manager.cc:5484] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:41.946653 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:41.953686 13917 tablet.cc:1722] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:41.954006 13917 tablet.cc:1620] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:41.954110 13919 tablet.cc:1722] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:41.954195 13918 tablet.cc:1722] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:41.954389 13919 tablet.cc:1620] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:41.954509 13918 tablet.cc:1620] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:41.960510 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=05080a15410a41799c116600c206843d] alter complete (version 2)
I20240426 09:49:42.007323 12362 raft_consensus.cc:482] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:42.007444 12362 raft_consensus.cc:504] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.007831 12362 leader_election.cc:288] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:42.008198 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6112d2390c994fefb74b8e7f75d57d17" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:42.008198 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6112d2390c994fefb74b8e7f75d57d17" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:42.008327 9286 raft_consensus.cc:2454] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:42.008342 9416 raft_consensus.cc:2454] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:42.008651 9614 leader_election.cc:302] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:42.008812 12362 raft_consensus.cc:2781] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:42.008908 12362 raft_consensus.cc:482] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:42.008955 12362 raft_consensus.cc:3037] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.009793 12362 raft_consensus.cc:504] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.010164 12362 leader_election.cc:288] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:42.010457 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6112d2390c994fefb74b8e7f75d57d17" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:42.010483 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6112d2390c994fefb74b8e7f75d57d17" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:42.010576 9416 raft_consensus.cc:3037] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.010576 9286 raft_consensus.cc:3037] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.011548 9286 raft_consensus.cc:2454] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:42.011548 9416 raft_consensus.cc:2454] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:42.011813 9614 leader_election.cc:302] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:42.011981 12362 raft_consensus.cc:2781] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:42.012075 12362 raft_consensus.cc:686] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:42.012215 12362 consensus_queue.cc:237] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.014009 6738 catalog_manager.cc:5484] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:42.036753 6993 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:42.041733 9397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:42.046619 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:42.046985 9266 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:42.051578 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:42.059361 9660 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:42.060107 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:42.063843 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:42.069958 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:42.084842 13917 tablet.cc:1722] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:42.087678 13919 tablet.cc:1722] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:42.088536 13918 tablet.cc:1722] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:42.094179 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=05080a15410a41799c116600c206843d] alter complete (version 3)
I20240426 09:49:42.105006 13975 ts_tablet_manager.cc:902] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:42.105551 13975 tablet_copy_client.cc:308] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:42.105957 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 4189112964fb425aac074352dd873415 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:41455)
I20240426 09:49:42.106066 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 4189112964fb425aac074352dd873415 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:41455: session id = 558c4ecd0ece460d9591aa7a3c799d19-4189112964fb425aac074352dd873415
I20240426 09:49:42.106827 7295 tablet_copy_source_session.cc:205] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:42.107640 13975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4189112964fb425aac074352dd873415. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.109319 13975 tablet_copy_client.cc:783] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:42.109467 13975 tablet_copy_client.cc:647] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:42.111021 13975 tablet_copy_client.cc:523] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:42.112253 13975 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:42.120802 9549 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cd67a40e0b834a83b41259ee4da3f3db"
dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:42860
I20240426 09:49:42.120999 9549 raft_consensus.cc:595] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:42.136018 13975 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:42.136390 13975 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:42.136598 13975 ts_tablet_manager.cc:1367] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.024s user 0.024s sys 0.000s
I20240426 09:49:42.137168 13975 raft_consensus.cc:348] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.137328 13975 raft_consensus.cc:729] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.137468 13975 consensus_queue.cc:260] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.137934 13975 ts_tablet_manager.cc:1398] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240426 09:49:42.140656 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-4189112964fb425aac074352dd873415 received from {username='slave'} at 127.0.99.136:41455
I20240426 09:49:42.140745 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-4189112964fb425aac074352dd873415 on tablet 4189112964fb425aac074352dd873415 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:42.167737 14022 ts_tablet_manager.cc:902] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:42.168748 14022 tablet_copy_client.cc:308] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:42.169728 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 948e1df8d52f42cebcd289fff377d6fb from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:54839)
I20240426 09:49:42.169847 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 948e1df8d52f42cebcd289fff377d6fb from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:54839: session id = a692602e1a2e4794877e6fb92a7b9962-948e1df8d52f42cebcd289fff377d6fb
I20240426 09:49:42.170836 6902 tablet_copy_source_session.cc:205] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:42.172163 14022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948e1df8d52f42cebcd289fff377d6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.177064 14022 tablet_copy_client.cc:783] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:42.178160 14022 tablet_copy_client.cc:647] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:42.179361 14022 tablet_copy_client.cc:523] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:42.181133 14022 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:42.197571 6739 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:42.197819 6739 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:42.200181 6739 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 24146d69f1a147e7933836f4b44b41bf
I20240426 09:49:42.200697 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 24146d69f1a147e7933836f4b44b41bf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:42 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:42.200960 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 24146d69f1a147e7933836f4b44b41bf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:42 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:42.200960 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 24146d69f1a147e7933836f4b44b41bf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:42 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:42.203123 14029 tablet_replica.cc:330] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:42.203361 14028 tablet_replica.cc:330] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:42.203564 14028 raft_consensus.cc:2229] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:42.203794 14028 raft_consensus.cc:2258] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:42.204032 14029 raft_consensus.cc:2229] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:42.204270 14029 raft_consensus.cc:2258] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:42.204329 14028 ts_tablet_manager.cc:1875] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:42.204607 14030 tablet_replica.cc:330] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:42.205057 14029 ts_tablet_manager.cc:1875] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:42.206751 14028 ts_tablet_manager.cc:1888] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:42.206887 14028 log.cc:1196] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/24146d69f1a147e7933836f4b44b41bf
I20240426 09:49:42.207006 14030 raft_consensus.cc:2229] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:42.207252 14028 ts_tablet_manager.cc:1909] T 24146d69f1a147e7933836f4b44b41bf P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:42.207253 14030 raft_consensus.cc:2258] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:42.207852 14029 ts_tablet_manager.cc:1888] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:42.207963 14029 log.cc:1196] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/24146d69f1a147e7933836f4b44b41bf
I20240426 09:49:42.208050 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 24146d69f1a147e7933836f4b44b41bf (table rebalancer_dynamic_table_new_name [id=05080a15410a41799c116600c206843d]) successfully deleted
I20240426 09:49:42.208343 14029 ts_tablet_manager.cc:1909] T 24146d69f1a147e7933836f4b44b41bf P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:42.208598 14030 ts_tablet_manager.cc:1875] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:42.209076 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 24146d69f1a147e7933836f4b44b41bf (table rebalancer_dynamic_table_new_name [id=05080a15410a41799c116600c206843d]) successfully deleted
I20240426 09:49:42.213654 14030 ts_tablet_manager.cc:1888] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:42.214350 14030 log.cc:1196] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/24146d69f1a147e7933836f4b44b41bf
I20240426 09:49:42.214664 14030 ts_tablet_manager.cc:1909] T 24146d69f1a147e7933836f4b44b41bf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:42.218713 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 24146d69f1a147e7933836f4b44b41bf (table rebalancer_dynamic_table_new_name [id=05080a15410a41799c116600c206843d]) successfully deleted
I20240426 09:49:42.221931 14022 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:42.222397 14022 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:42.222599 14022 ts_tablet_manager.cc:1367] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.042s user 0.027s sys 0.000s
I20240426 09:49:42.223501 14022 raft_consensus.cc:348] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.223732 14022 raft_consensus.cc:729] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.223949 14022 consensus_queue.cc:260] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.225303 14022 ts_tablet_manager.cc:1398] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:42.225847 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-948e1df8d52f42cebcd289fff377d6fb received from {username='slave'} at 127.0.99.133:54839
I20240426 09:49:42.225936 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-948e1df8d52f42cebcd289fff377d6fb on tablet 948e1df8d52f42cebcd289fff377d6fb with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:42.236352 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:42.237097 9660 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:42.240060 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:42.244462 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:42.246943 6993 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:42.247574 9266 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:42.251121 9397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:42.257367 12476 raft_consensus.cc:979] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: : Instructing follower 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:49:42.257455 12476 raft_consensus.cc:1067] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Signalling peer 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:49:42.258049 7013 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cd67a40e0b834a83b41259ee4da3f3db"
dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
from {username='slave'} at 127.0.99.135:42099
I20240426 09:49:42.258184 7013 raft_consensus.cc:482] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:42.258234 7013 raft_consensus.cc:3037] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:42.259296 7013 raft_consensus.cc:504] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.259699 7013 leader_election.cc:288] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:42.260373 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd67a40e0b834a83b41259ee4da3f3db" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:42.260489 7143 raft_consensus.cc:3037] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:42.261499 7143 raft_consensus.cc:2454] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:49:42.261756 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd67a40e0b834a83b41259ee4da3f3db" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:42.261864 9672 raft_consensus.cc:3037] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:42.262818 9672 raft_consensus.cc:2454] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:49:42.263283 6946 leader_election.cc:302] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:42.263475 13601 raft_consensus.cc:2781] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:42.263551 13601 raft_consensus.cc:686] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:42.263684 13601 consensus_queue.cc:237] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.264864 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd67a40e0b834a83b41259ee4da3f3db" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:42.264988 9549 raft_consensus.cc:3032] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:42.265026 9549 raft_consensus.cc:729] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:42.265166 9549 consensus_queue.cc:260] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.265339 9549 raft_consensus.cc:3037] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:42.266230 9549 raft_consensus.cc:2454] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:49:42.272722 6731 catalog_manager.cc:5484] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 1 to 2, leader changed from b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:42.289803 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:42.302541 6731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:42.308044 9397 tablet_service.cc:1467] Processing CreateTablet for tablet cf2a3fce165b450baf767579d6a7043c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=79364a827593482faeb3a63819def707]), partition=
I20240426 09:49:42.308162 9266 tablet_service.cc:1467] Processing CreateTablet for tablet cf2a3fce165b450baf767579d6a7043c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=79364a827593482faeb3a63819def707]), partition=
I20240426 09:49:42.308183 9527 tablet_service.cc:1467] Processing CreateTablet for tablet cf2a3fce165b450baf767579d6a7043c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=79364a827593482faeb3a63819def707]), partition=
I20240426 09:49:42.308317 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2a3fce165b450baf767579d6a7043c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.308364 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2a3fce165b450baf767579d6a7043c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.308420 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf2a3fce165b450baf767579d6a7043c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.310271 13855 tablet_bootstrap.cc:492] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:42.310328 13904 tablet_bootstrap.cc:492] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:42.311218 13904 tablet_bootstrap.cc:654] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.311347 13855 tablet_bootstrap.cc:654] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.312551 13855 tablet_bootstrap.cc:492] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:42.312637 13855 ts_tablet_manager.cc:1367] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:42.312683 13904 tablet_bootstrap.cc:492] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:42.312752 13904 ts_tablet_manager.cc:1367] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240426 09:49:42.313199 13904 raft_consensus.cc:348] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.313288 13904 raft_consensus.cc:374] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.313246 13855 raft_consensus.cc:348] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.313310 13904 raft_consensus.cc:729] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.313328 13855 raft_consensus.cc:374] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.313390 13855 raft_consensus.cc:729] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.313424 13904 consensus_queue.cc:260] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.313503 13855 consensus_queue.cc:260] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.313649 13904 ts_tablet_manager.cc:1398] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240426 09:49:42.313750 13855 ts_tablet_manager.cc:1398] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:42.315845 14043 tablet_bootstrap.cc:492] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:42.316782 14043 tablet_bootstrap.cc:654] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.318580 14043 tablet_bootstrap.cc:492] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:42.318651 14043 ts_tablet_manager.cc:1367] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:42.319146 14043 raft_consensus.cc:348] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.319290 14043 raft_consensus.cc:374] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.319382 14043 raft_consensus.cc:729] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.319510 14043 consensus_queue.cc:260] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.319753 14043 ts_tablet_manager.cc:1398] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:42.326802 12388 raft_consensus.cc:482] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:42.326895 12388 raft_consensus.cc:504] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.327232 12388 leader_election.cc:288] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:42.327701 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2a3fce165b450baf767579d6a7043c" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:42.327843 9549 raft_consensus.cc:2454] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:42.328053 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2a3fce165b450baf767579d6a7043c" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:42.328167 9416 raft_consensus.cc:2454] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:42.328424 9219 leader_election.cc:302] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:42.329236 12388 raft_consensus.cc:2781] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:42.329303 12388 raft_consensus.cc:482] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:42.329339 12388 raft_consensus.cc:3037] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.330335 12388 raft_consensus.cc:504] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.330639 12388 leader_election.cc:288] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:42.330919 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2a3fce165b450baf767579d6a7043c" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:42.331018 9549 raft_consensus.cc:3037] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.330991 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf2a3fce165b450baf767579d6a7043c" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:42.331104 9416 raft_consensus.cc:3037] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.332006 9549 raft_consensus.cc:2454] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:42.332006 9416 raft_consensus.cc:2454] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:42.332451 9219 leader_election.cc:302] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:42.332878 12388 raft_consensus.cc:2781] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:42.332962 12388 raft_consensus.cc:686] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:42.333082 12388 consensus_queue.cc:237] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.335194 6731 catalog_manager.cc:5484] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:42.379601 13787 consensus_queue.cc:1035] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:42.385493 12388 consensus_queue.cc:1035] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:42.388119 12838 consensus_queue.cc:1035] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:42.397679 13669 consensus_queue.cc:1035] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:42.443542 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:42.447969 6993 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:42.450496 6862 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:42.453244 9527 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:42.456743 12362 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:42.457496 9266 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:42.461256 9416 raft_consensus.cc:1261] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:42.463100 12388 consensus_queue.cc:1035] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:42.464010 13321 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:42.466717 9397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:42.466859 6731 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet cf2a3fce165b450baf767579d6a7043c (table rebalancer_dynamic_table [id=79364a827593482faeb3a63819def707]). Expected version 1 got 0
I20240426 09:49:42.467952 9549 raft_consensus.cc:1261] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:42.468360 12388 consensus_queue.cc:1035] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:42.471129 6734 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet cf2a3fce165b450baf767579d6a7043c (table rebalancer_dynamic_table [id=79364a827593482faeb3a63819def707]). Expected version 1 got 0
I20240426 09:49:42.475769 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:42.480779 13917 tablet.cc:1722] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:42.481122 13917 tablet.cc:1620] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:42.482100 13918 tablet.cc:1722] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:42.482414 13918 tablet.cc:1620] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:42.485983 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=79364a827593482faeb3a63819def707] alter complete (version 1)
I20240426 09:49:42.489010 9660 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:42.490172 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:42.525159 14121 tablet.cc:1722] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:42.525626 14121 tablet.cc:1620] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:42.526510 6734 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet cf2a3fce165b450baf767579d6a7043c (table rebalancer_dynamic_table [id=79364a827593482faeb3a63819def707]). Expected version 1 got 0
I20240426 09:49:42.558966 7143 consensus_queue.cc:237] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:42.560279 9286 raft_consensus.cc:1261] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:42.560284 7274 raft_consensus.cc:1261] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:42.560755 13344 consensus_queue.cc:1035] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:42.560904 12903 consensus_queue.cc:1035] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:42.562492 12903 raft_consensus.cc:2932] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:42.562955 7274 raft_consensus.cc:2932] T d0779912d7a04b4c95dd1db6f9c4d6fd P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:42.565270 6734 catalog_manager.cc:5484] T d0779912d7a04b4c95dd1db6f9c4d6fd P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:42.565325 9286 raft_consensus.cc:2932] T d0779912d7a04b4c95dd1db6f9c4d6fd P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:42.567629 6731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:42.574488 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet d0779912d7a04b4c95dd1db6f9c4d6fd with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:42.574754 14029 tablet_replica.cc:330] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:42.574939 14029 raft_consensus.cc:2229] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:42.575063 14029 raft_consensus.cc:2258] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:42.575529 14029 ts_tablet_manager.cc:1875] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:42.576758 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 22930f3d5b514c9884359f6c939c53e3 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:42.576879 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 22930f3d5b514c9884359f6c939c53e3 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:42.577005 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22930f3d5b514c9884359f6c939c53e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.577155 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22930f3d5b514c9884359f6c939c53e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.577764 14029 ts_tablet_manager.cc:1888] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:42.577821 14029 log.cc:1196] T d0779912d7a04b4c95dd1db6f9c4d6fd P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/d0779912d7a04b4c95dd1db6f9c4d6fd
I20240426 09:49:42.578787 7123 tablet_service.cc:1467] Processing CreateTablet for tablet f1bb1cdfa9684023ad017c0e36cb44b1 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:42.578925 14145 tablet_bootstrap.cc:492] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:42.579061 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1bb1cdfa9684023ad017c0e36cb44b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.579241 9397 tablet_service.cc:1467] Processing CreateTablet for tablet c6eb1eaad9ac4dc9aa94739e482ebd9c (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:42.579461 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet d0779912d7a04b4c95dd1db6f9c4d6fd (table rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]) successfully deleted
I20240426 09:49:42.579506 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6eb1eaad9ac4dc9aa94739e482ebd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.579595 9660 tablet_service.cc:1467] Processing CreateTablet for tablet c6eb1eaad9ac4dc9aa94739e482ebd9c (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:42.579789 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6eb1eaad9ac4dc9aa94739e482ebd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.581456 14146 tablet_bootstrap.cc:492] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:42.581663 13904 tablet_bootstrap.cc:492] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:42.581794 9527 tablet_service.cc:1467] Processing CreateTablet for tablet f1bb1cdfa9684023ad017c0e36cb44b1 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:42.582051 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1bb1cdfa9684023ad017c0e36cb44b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.582197 14145 tablet_bootstrap.cc:654] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.582684 13904 tablet_bootstrap.cc:654] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.583307 14145 tablet_bootstrap.cc:492] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:42.583376 14145 ts_tablet_manager.cc:1367] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:42.583793 14145 raft_consensus.cc:348] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.583890 14145 raft_consensus.cc:374] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.583839 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 22930f3d5b514c9884359f6c939c53e3 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:42.583920 14145 raft_consensus.cc:729] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.584030 14146 tablet_bootstrap.cc:654] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.584079 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22930f3d5b514c9884359f6c939c53e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.584024 14145 consensus_queue.cc:260] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.578791 7122 tablet_service.cc:1467] Processing CreateTablet for tablet c6eb1eaad9ac4dc9aa94739e482ebd9c (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:42.584300 14145 ts_tablet_manager.cc:1398] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:42.584451 14145 tablet_bootstrap.cc:492] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:42.584471 14043 tablet_bootstrap.cc:492] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:42.585261 14145 tablet_bootstrap.cc:654] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.585335 14043 tablet_bootstrap.cc:654] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.586253 14043 tablet_bootstrap.cc:492] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:42.586326 14043 ts_tablet_manager.cc:1367] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:42.586310 6860 tablet_service.cc:1467] Processing CreateTablet for tablet f1bb1cdfa9684023ad017c0e36cb44b1 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:42.586515 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1bb1cdfa9684023ad017c0e36cb44b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.586556 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6eb1eaad9ac4dc9aa94739e482ebd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.586753 14043 raft_consensus.cc:348] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:42.586915 14043 raft_consensus.cc:374] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.586946 14043 raft_consensus.cc:729] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.587038 14043 consensus_queue.cc:260] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:42.587280 14043 ts_tablet_manager.cc:1398] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:42.588395 14145 tablet_bootstrap.cc:492] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:42.588460 14145 ts_tablet_manager.cc:1367] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:42.588944 9672 raft_consensus.cc:1203] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:42.589128 14145 raft_consensus.cc:348] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.589864 14145 raft_consensus.cc:374] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.589938 14145 raft_consensus.cc:729] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.590103 14145 consensus_queue.cc:260] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.590471 14145 ts_tablet_manager.cc:1398] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:42.590899 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:42.591722 14147 tablet_bootstrap.cc:492] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:42.594844 14153 tablet_bootstrap.cc:492] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:42.595717 13904 tablet_bootstrap.cc:492] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:42.595809 13904 ts_tablet_manager.cc:1367] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20240426 09:49:42.596324 13904 raft_consensus.cc:348] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.596428 13904 raft_consensus.cc:374] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.596458 13904 raft_consensus.cc:729] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.596565 13904 consensus_queue.cc:260] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.596841 14147 tablet_bootstrap.cc:654] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.596863 13904 ts_tablet_manager.cc:1398] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:42.597429 14153 tablet_bootstrap.cc:654] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.601068 13917 tablet.cc:1722] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:42.601435 13917 tablet.cc:1620] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:42.601634 14121 tablet.cc:1722] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:42.601903 14121 tablet.cc:1620] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:42.603066 13918 tablet.cc:1722] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:42.603375 13918 tablet.cc:1620] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:42.603874 12362 raft_consensus.cc:482] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:42.603958 12362 raft_consensus.cc:504] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.604418 12362 leader_election.cc:288] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:42.604867 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6eb1eaad9ac4dc9aa94739e482ebd9c" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:42.604979 9416 raft_consensus.cc:2454] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:42.605268 9614 leader_election.cc:302] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:42.605953 14153 tablet_bootstrap.cc:492] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:42.606051 14153 ts_tablet_manager.cc:1367] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:42.606257 12362 raft_consensus.cc:2781] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:42.606308 12362 raft_consensus.cc:482] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:42.606341 12362 raft_consensus.cc:3037] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.606528 14153 raft_consensus.cc:348] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.606643 14153 raft_consensus.cc:374] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.606676 14153 raft_consensus.cc:729] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.606853 14153 consensus_queue.cc:260] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.607347 12362 raft_consensus.cc:504] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.607467 14153 ts_tablet_manager.cc:1398] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:42.607683 12362 leader_election.cc:288] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:42.608390 14153 tablet_bootstrap.cc:492] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:42.609444 14153 tablet_bootstrap.cc:654] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.611032 14147 tablet_bootstrap.cc:492] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:42.611214 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=79364a827593482faeb3a63819def707] alter complete (version 2)
I20240426 09:49:42.611191 14147 ts_tablet_manager.cc:1367] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20240426 09:49:42.611814 14147 raft_consensus.cc:348] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:42.611958 14147 raft_consensus.cc:374] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.612012 14147 raft_consensus.cc:729] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.612164 14147 consensus_queue.cc:260] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:42.612483 14147 ts_tablet_manager.cc:1398] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:42.612708 14147 tablet_bootstrap.cc:492] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:42.613837 14147 tablet_bootstrap.cc:654] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.614832 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6eb1eaad9ac4dc9aa94739e482ebd9c" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:42.614959 9416 raft_consensus.cc:3037] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.615722 14153 tablet_bootstrap.cc:492] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:42.615811 14153 ts_tablet_manager.cc:1367] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:42.615940 9416 raft_consensus.cc:2454] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:42.616271 14153 raft_consensus.cc:348] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:42.616385 14153 raft_consensus.cc:374] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.616411 14153 raft_consensus.cc:729] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.616523 14153 consensus_queue.cc:260] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:42.616796 14153 ts_tablet_manager.cc:1398] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:42.618893 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6eb1eaad9ac4dc9aa94739e482ebd9c" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:42.619020 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6eb1eaad9ac4dc9aa94739e482ebd9c" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
W20240426 09:49:42.619282 9614 leader_election.cc:341] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Illegal state: must be running to vote when last-logged opid is not known
W20240426 09:49:42.619526 9614 leader_election.cc:341] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:42.619837 9614 leader_election.cc:302] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters: 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:42.620071 13321 raft_consensus.cc:2781] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:42.620144 13321 raft_consensus.cc:686] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:42.620261 13321 consensus_queue.cc:237] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.620704 14146 tablet_bootstrap.cc:492] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:42.620810 14146 ts_tablet_manager.cc:1367] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20240426 09:49:42.621299 14146 raft_consensus.cc:348] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.621421 14146 raft_consensus.cc:374] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.621452 14146 raft_consensus.cc:729] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.621608 14146 consensus_queue.cc:260] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.621956 14146 ts_tablet_manager.cc:1398] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:42.622319 6734 catalog_manager.cc:5484] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:42.624483 14147 tablet_bootstrap.cc:492] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:42.624580 14147 ts_tablet_manager.cc:1367] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240426 09:49:42.625056 14147 raft_consensus.cc:348] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.625164 14147 raft_consensus.cc:374] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.625195 14147 raft_consensus.cc:729] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.625312 14147 consensus_queue.cc:260] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.625536 14147 ts_tablet_manager.cc:1398] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:42.705590 7142 raft_consensus.cc:1261] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:49:42.706017 12905 consensus_queue.cc:1035] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:42.715402 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:42.721230 13917 tablet.cc:1722] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:42.721581 14121 tablet.cc:1722] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:42.722368 9672 raft_consensus.cc:1261] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:49:42.722826 12905 consensus_queue.cc:1035] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:42.723106 13918 tablet.cc:1722] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:42.728349 9548 raft_consensus.cc:1261] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:49:42.728821 12905 consensus_queue.cc:1035] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:42.732692 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=79364a827593482faeb3a63819def707] alter complete (version 3)
I20240426 09:49:42.750620 13758 raft_consensus.cc:482] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:42.750735 13758 raft_consensus.cc:504] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:42.751405 13758 leader_election.cc:288] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:42.751916 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bb1cdfa9684023ad017c0e36cb44b1" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:42.752048 6882 raft_consensus.cc:2454] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:42.752454 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bb1cdfa9684023ad017c0e36cb44b1" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:42.752584 7142 raft_consensus.cc:2454] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:42.752928 9484 leader_election.cc:302] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:42.753615 13758 raft_consensus.cc:2781] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:42.753669 13758 raft_consensus.cc:482] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:42.753700 13758 raft_consensus.cc:3037] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.754777 13758 raft_consensus.cc:504] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:42.755125 13758 leader_election.cc:288] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:42.755697 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bb1cdfa9684023ad017c0e36cb44b1" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:42.755849 6882 raft_consensus.cc:3037] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.756541 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1bb1cdfa9684023ad017c0e36cb44b1" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:42.756664 7142 raft_consensus.cc:3037] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.757140 6882 raft_consensus.cc:2454] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:42.757417 9484 leader_election.cc:302] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:42.757535 7142 raft_consensus.cc:2454] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:42.757601 13758 raft_consensus.cc:2781] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:42.757666 13758 raft_consensus.cc:686] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:42.757779 13758 consensus_queue.cc:237] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:42.759755 6732 catalog_manager.cc:5484] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:42.761611 13815 raft_consensus.cc:482] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:42.761696 13815 raft_consensus.cc:504] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.762027 13815 leader_election.cc:288] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:42.762961 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22930f3d5b514c9884359f6c939c53e3" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:42.763087 9672 raft_consensus.cc:2454] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:42.763410 6815 leader_election.cc:302] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:42.763620 13815 raft_consensus.cc:2781] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:42.763672 13815 raft_consensus.cc:482] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:42.763708 13815 raft_consensus.cc:3037] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.764703 13815 raft_consensus.cc:504] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.765002 13815 leader_election.cc:288] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:42.765511 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22930f3d5b514c9884359f6c939c53e3" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:42.765623 7011 raft_consensus.cc:3037] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.766561 7011 raft_consensus.cc:2454] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:42.767484 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22930f3d5b514c9884359f6c939c53e3" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:42.767598 9672 raft_consensus.cc:3037] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.768489 9672 raft_consensus.cc:2454] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:42.769037 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22930f3d5b514c9884359f6c939c53e3" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:42.769174 7013 raft_consensus.cc:2362] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1. Re-sending same reply.
I20240426 09:49:42.769381 6815 leader_election.cc:302] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:42.769538 13815 raft_consensus.cc:2781] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:42.769716 13815 raft_consensus.cc:686] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:42.769867 13815 consensus_queue.cc:237] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:42.772312 6738 catalog_manager.cc:5484] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:42.773162 9286 raft_consensus.cc:1203] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:42.789438 6860 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:42.793282 6993 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:42.796311 9397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:42.796965 13344 raft_consensus.cc:482] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:42.797075 13344 raft_consensus.cc:504] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:42.797462 13344 leader_election.cc:288] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:42.798087 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6eb1eaad9ac4dc9aa94739e482ebd9c" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:42.799058 9660 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:42.799154 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6eb1eaad9ac4dc9aa94739e482ebd9c" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:42.799276 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:42.799325 9416 raft_consensus.cc:2379] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1aaf8befba984e199e96f53f46b50441 in current term 1: Already voted for candidate 558c4ecd0ece460d9591aa7a3c799d19 in this term.
I20240426 09:49:42.799624 7077 leader_election.cc:302] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441; no voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:42.799899 13344 raft_consensus.cc:3037] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.800979 13344 raft_consensus.cc:2726] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240426 09:49:42.813184 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:42.815508 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:42.816864 9266 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:42.834569 6738 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:42.834740 6738 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:42.837750 6738 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet cf2a3fce165b450baf767579d6a7043c
I20240426 09:49:42.838904 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2a3fce165b450baf767579d6a7043c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:42 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:42.839227 14268 tablet_replica.cc:330] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:42.839519 14268 raft_consensus.cc:2229] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:42.839689 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2a3fce165b450baf767579d6a7043c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:42 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:42.839819 14268 raft_consensus.cc:2258] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:42.840315 14268 ts_tablet_manager.cc:1875] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:42.842783 14268 ts_tablet_manager.cc:1888] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:42.842939 14268 log.cc:1196] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/cf2a3fce165b450baf767579d6a7043c
I20240426 09:49:42.843319 14268 ts_tablet_manager.cc:1909] T cf2a3fce165b450baf767579d6a7043c P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:42.844331 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet cf2a3fce165b450baf767579d6a7043c (table rebalancer_dynamic_table_new_name [id=79364a827593482faeb3a63819def707]) successfully deleted
I20240426 09:49:42.846921 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet cf2a3fce165b450baf767579d6a7043c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:42 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:42.847285 14270 tablet_replica.cc:330] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:42.847642 14270 raft_consensus.cc:2229] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:42.847781 14270 raft_consensus.cc:2258] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:42.848807 14270 ts_tablet_manager.cc:1875] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:42.852944 14270 ts_tablet_manager.cc:1888] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:42.853107 14270 log.cc:1196] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/cf2a3fce165b450baf767579d6a7043c
I20240426 09:49:42.853478 14270 ts_tablet_manager.cc:1909] T cf2a3fce165b450baf767579d6a7043c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:42.854276 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet cf2a3fce165b450baf767579d6a7043c (table rebalancer_dynamic_table_new_name [id=79364a827593482faeb3a63819def707]) successfully deleted
I20240426 09:49:42.854422 14269 tablet_replica.cc:330] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:42.854653 14269 raft_consensus.cc:2229] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:42.854828 14269 raft_consensus.cc:2258] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:42.855511 14269 ts_tablet_manager.cc:1875] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:42.857758 14269 ts_tablet_manager.cc:1888] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:42.857863 14269 log.cc:1196] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/cf2a3fce165b450baf767579d6a7043c
I20240426 09:49:42.858156 14269 ts_tablet_manager.cc:1909] T cf2a3fce165b450baf767579d6a7043c P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:42.858842 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet cf2a3fce165b450baf767579d6a7043c (table rebalancer_dynamic_table_new_name [id=79364a827593482faeb3a63819def707]) successfully deleted
I20240426 09:49:42.885196 6882 consensus_queue.cc:237] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.886395 9286 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:42.886564 7274 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:42.886754 13815 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:42.886929 14255 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:42.888494 13815 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:42.889307 7274 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:42.890219 9286 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:42.892383 6732 catalog_manager.cc:5484] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:42.897243 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 948e1df8d52f42cebcd289fff377d6fb with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:42.897732 14294 tablet_replica.cc:330] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:42.897929 14294 raft_consensus.cc:2229] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:42.898185 14294 raft_consensus.cc:2258] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:42.898684 14294 ts_tablet_manager.cc:1875] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:42.901084 14294 ts_tablet_manager.cc:1888] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:42.901170 14294 log.cc:1196] T 948e1df8d52f42cebcd289fff377d6fb P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/948e1df8d52f42cebcd289fff377d6fb
I20240426 09:49:42.901852 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 948e1df8d52f42cebcd289fff377d6fb (table rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]) successfully deleted
I20240426 09:49:42.931375 6882 consensus_queue.cc:237] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:42.933142 7274 raft_consensus.cc:1261] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:42.933279 7013 raft_consensus.cc:1261] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:42.933665 14316 consensus_queue.cc:1035] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:49:42.933882 6815 consensus_peers.cc:485] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a42302cab4524e7a85defe4da40be268. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:42.933986 14316 consensus_queue.cc:1035] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:42.935935 13815 raft_consensus.cc:2932] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:42.936344 7013 raft_consensus.cc:2932] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:42.939587 6738 catalog_manager.cc:5484] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:42.939589 7274 raft_consensus.cc:2932] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:42.940357 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:42.949102 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 2a59c860a09f48de97556e93b172b9ff (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d650f95ebfd94e9eb6a5d93819e13074]), partition=
I20240426 09:49:42.949386 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a59c860a09f48de97556e93b172b9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.949355 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 2a59c860a09f48de97556e93b172b9ff (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d650f95ebfd94e9eb6a5d93819e13074]), partition=
I20240426 09:49:42.949582 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a59c860a09f48de97556e93b172b9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.952615 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 2a59c860a09f48de97556e93b172b9ff (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d650f95ebfd94e9eb6a5d93819e13074]), partition=
I20240426 09:49:42.952885 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a59c860a09f48de97556e93b172b9ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:42.953338 14043 tablet_bootstrap.cc:492] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:42.954645 14043 tablet_bootstrap.cc:654] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.955798 14043 tablet_bootstrap.cc:492] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:42.955914 14043 ts_tablet_manager.cc:1367] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:42.956440 14043 raft_consensus.cc:348] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.956583 14043 raft_consensus.cc:374] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.956637 14043 raft_consensus.cc:729] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.956764 14043 consensus_queue.cc:260] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.957062 14043 ts_tablet_manager.cc:1398] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:42.963001 14320 tablet_bootstrap.cc:492] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:42.963006 14326 tablet_bootstrap.cc:492] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:42.964272 14320 tablet_bootstrap.cc:654] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.964587 14326 tablet_bootstrap.cc:654] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:42.966723 14320 tablet_bootstrap.cc:492] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:42.966882 14320 ts_tablet_manager.cc:1367] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:42.967303 14326 tablet_bootstrap.cc:492] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:42.967409 14326 ts_tablet_manager.cc:1367] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:42.968047 14326 raft_consensus.cc:348] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.968063 14320 raft_consensus.cc:348] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.968175 14320 raft_consensus.cc:374] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.968175 14326 raft_consensus.cc:374] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:42.968200 14320 raft_consensus.cc:729] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.968214 14326 raft_consensus.cc:729] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:42.968341 14320 consensus_queue.cc:260] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.968359 14326 consensus_queue.cc:260] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.968647 14326 ts_tablet_manager.cc:1398] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:42.969395 14320 ts_tablet_manager.cc:1398] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:42.969481 13916 raft_consensus.cc:482] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:42.969555 13916 raft_consensus.cc:504] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.969873 13916 leader_election.cc:288] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:42.970163 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a59c860a09f48de97556e93b172b9ff" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:42.970289 7274 raft_consensus.cc:2454] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:42.970830 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a59c860a09f48de97556e93b172b9ff" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:42.970983 9548 raft_consensus.cc:2454] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:42.971091 9221 leader_election.cc:302] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:42.971645 13916 raft_consensus.cc:2781] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:42.971704 13916 raft_consensus.cc:482] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:42.971741 13916 raft_consensus.cc:3037] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.972720 13916 raft_consensus.cc:504] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.973073 13916 leader_election.cc:288] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:42.973387 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a59c860a09f48de97556e93b172b9ff" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:42.973520 9548 raft_consensus.cc:3037] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.973851 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a59c860a09f48de97556e93b172b9ff" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:42.973955 7274 raft_consensus.cc:3037] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:42.974609 9548 raft_consensus.cc:2454] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:42.974920 9219 leader_election.cc:302] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:42.974964 7274 raft_consensus.cc:2454] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:42.975100 13916 raft_consensus.cc:2781] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:42.975173 13916 raft_consensus.cc:686] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:42.975292 13916 consensus_queue.cc:237] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:42.977159 6737 catalog_manager.cc:5484] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:42.990131 7142 consensus_queue.cc:237] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:42.991565 7274 raft_consensus.cc:1261] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:42.991580 7013 raft_consensus.cc:1261] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:42.992082 7077 consensus_peers.cc:485] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f834d8a08b1341869fd644ff400fceae. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:42.992120 14342 consensus_queue.cc:1035] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:42.992422 12903 consensus_queue.cc:1035] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:42.994354 12903 raft_consensus.cc:2932] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:42.994813 7274 raft_consensus.cc:2932] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:42.997040 7013 raft_consensus.cc:2932] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:42.998411 6737 catalog_manager.cc:5484] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:43.018311 9669 consensus_queue.cc:237] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:43.019740 9416 raft_consensus.cc:1261] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:43.019871 9614 consensus_peers.cc:485] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdcd6105a2647738257638e1e1096c3. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:43.019887 9286 raft_consensus.cc:1261] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:43.020188 14361 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:43.020354 13321 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:43.022300 14361 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:43.022900 9286 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:43.025159 9416 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:43.025485 6731 catalog_manager.cc:5484] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:43.044489 9669 consensus_queue.cc:237] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.045935 9548 raft_consensus.cc:1261] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:43.045975 7274 raft_consensus.cc:1261] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:43.046519 9614 consensus_peers.cc:485] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e856ff2c8ffb4bd4b44c694047c8a8fa. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:43.046520 12362 consensus_queue.cc:1035] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:43.046929 13321 consensus_queue.cc:1035] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:43.048485 12362 raft_consensus.cc:2932] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:43.048918 9548 raft_consensus.cc:2932] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:43.050375 7274 raft_consensus.cc:2932] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:43.051676 6738 catalog_manager.cc:5484] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:43.072628 7274 consensus_queue.cc:237] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:43.074079 7013 raft_consensus.cc:1261] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:43.074312 7142 raft_consensus.cc:1261] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:43.074330 7208 consensus_peers.cc:485] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4fb0f5d2d55a45599c1b09c5684522e6. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:43.074707 14398 consensus_queue.cc:1035] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:43.074939 13669 consensus_queue.cc:1035] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:43.076740 14398 raft_consensus.cc:2932] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:43.077155 7013 raft_consensus.cc:2932] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:43.079396 7142 raft_consensus.cc:2932] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:43.079933 6731 catalog_manager.cc:5484] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:43.082376 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:43.086145 7274 raft_consensus.cc:1261] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:43.086428 9548 raft_consensus.cc:1261] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:43.087071 13916 consensus_queue.cc:1035] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.087214 12388 consensus_queue.cc:1035] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.089705 6732 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet 2a59c860a09f48de97556e93b172b9ff (table rebalancer_dynamic_table [id=d650f95ebfd94e9eb6a5d93819e13074]). Expected version 1 got 0
I20240426 09:49:43.089789 13568 mvcc.cc:205] Tried to move back new op lower bound from 7021055930717163520 to 7021055930268094464. Current Snapshot: MvccSnapshot[applied={T|T < 7021055930717163520}]
I20240426 09:49:43.089880 13917 tablet.cc:1722] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:43.090237 13917 tablet.cc:1620] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:43.090251 6737 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 2a59c860a09f48de97556e93b172b9ff (table rebalancer_dynamic_table [id=d650f95ebfd94e9eb6a5d93819e13074]). Expected version 1 got 0
I20240426 09:49:43.092710 13918 tablet.cc:1722] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:43.093077 13918 tablet.cc:1620] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:43.096122 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=d650f95ebfd94e9eb6a5d93819e13074] alter complete (version 1)
I20240426 09:49:43.099408 14414 tablet.cc:1722] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:43.099771 14414 tablet.cc:1620] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:43.100618 6732 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 2a59c860a09f48de97556e93b172b9ff (table rebalancer_dynamic_table [id=d650f95ebfd94e9eb6a5d93819e13074]). Expected version 1 got 0
I20240426 09:49:43.110960 7142 consensus_queue.cc:237] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:43.112382 7013 raft_consensus.cc:1261] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:43.112375 6882 raft_consensus.cc:1261] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:43.112614 7077 consensus_peers.cc:485] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 639d1d6189fb4d5f8a2d0aa0d39575ec. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:43.112835 13344 consensus_queue.cc:1035] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:43.113003 12903 consensus_queue.cc:1035] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:43.114908 12903 raft_consensus.cc:2932] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:43.115365 7013 raft_consensus.cc:2932] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:43.116845 6882 raft_consensus.cc:2932] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:43.118402 6732 catalog_manager.cc:5484] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:43.138120 7013 consensus_queue.cc:237] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.139540 9548 raft_consensus.cc:1261] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:43.139611 9669 raft_consensus.cc:1261] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:43.139619 6947 consensus_peers.cc:485] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 852fad44c59f42dd9ce66b84819a9296. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:43.139945 12905 consensus_queue.cc:1035] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:43.140137 14245 consensus_queue.cc:1035] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:43.141952 12905 raft_consensus.cc:2932] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:43.142428 9548 raft_consensus.cc:2932] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:43.144198 9669 raft_consensus.cc:2932] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:43.145465 6737 catalog_manager.cc:5484] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:43.175318 7274 consensus_queue.cc:237] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:43.176918 7013 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:43.176918 7142 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:43.177359 14398 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:43.177521 13669 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:49:43.177810 7208 consensus_peers.cc:485] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 643fd2d359a14f8eaa8d8a0d0879ef61. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:43.179345 13669 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:43.179847 7013 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:43.182130 7142 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:43.183313 6738 catalog_manager.cc:5484] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:43.197638 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:43.199430 13321 consensus_queue.cc:1035] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.205789 13758 consensus_queue.cc:1035] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.206254 13917 tablet.cc:1722] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:43.206446 13918 tablet.cc:1722] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:43.206630 13917 tablet.cc:1620] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:43.206746 13918 tablet.cc:1620] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:43.207098 14361 consensus_queue.cc:1035] T c6eb1eaad9ac4dc9aa94739e482ebd9c P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.211796 14414 tablet.cc:1722] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:43.212113 14414 tablet.cc:1620] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:43.217131 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=d650f95ebfd94e9eb6a5d93819e13074] alter complete (version 2)
I20240426 09:49:43.217659 13758 consensus_queue.cc:1035] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.242089 14319 consensus_queue.cc:1035] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.247000 14255 consensus_queue.cc:1035] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.288906 6993 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:43.293095 9660 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:43.293140 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:43.295889 9397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240426 09:49:43.298405 6733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:43.304873 6860 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:43.306829 9266 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:43.314594 9265 tablet_service.cc:1467] Processing CreateTablet for tablet d1d91861708146eb94dcc2cdef1517eb (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:43.314929 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d91861708146eb94dcc2cdef1517eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.317101 14326 tablet_bootstrap.cc:492] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:43.318084 14326 tablet_bootstrap.cc:654] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.319226 9266 tablet_service.cc:1467] Processing CreateTablet for tablet aebef9b26fec446b8e71d4046768aedc (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:43.319478 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aebef9b26fec446b8e71d4046768aedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.322113 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 4d49693f3ade40f2a9d5a0029dd7508b (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:43.322228 9660 tablet_service.cc:1467] Processing CreateTablet for tablet aebef9b26fec446b8e71d4046768aedc (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:43.322417 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d49693f3ade40f2a9d5a0029dd7508b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.322494 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aebef9b26fec446b8e71d4046768aedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.324859 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 4d49693f3ade40f2a9d5a0029dd7508b (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:43.325102 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d49693f3ade40f2a9d5a0029dd7508b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.327482 9527 tablet_service.cc:1467] Processing CreateTablet for tablet d1d91861708146eb94dcc2cdef1517eb (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:43.327771 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d91861708146eb94dcc2cdef1517eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.327798 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:43.328202 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:43.330087 9395 tablet_service.cc:1467] Processing CreateTablet for tablet d1d91861708146eb94dcc2cdef1517eb (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:43.330336 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d91861708146eb94dcc2cdef1517eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.331142 14043 tablet_bootstrap.cc:492] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:43.332074 14326 tablet_bootstrap.cc:492] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:43.332180 14043 tablet_bootstrap.cc:654] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.332175 14326 ts_tablet_manager.cc:1367] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240426 09:49:43.332721 14326 raft_consensus.cc:348] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.332854 14326 raft_consensus.cc:374] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.332888 14326 raft_consensus.cc:729] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.333007 14326 consensus_queue.cc:260] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.330339 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 4d49693f3ade40f2a9d5a0029dd7508b (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:43.333297 14326 ts_tablet_manager.cc:1398] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.333468 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d49693f3ade40f2a9d5a0029dd7508b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.333514 14326 tablet_bootstrap.cc:492] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:43.334470 14326 tablet_bootstrap.cc:654] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.330035 9397 tablet_service.cc:1467] Processing CreateTablet for tablet aebef9b26fec446b8e71d4046768aedc (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:43.335450 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aebef9b26fec446b8e71d4046768aedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.337885 14326 tablet_bootstrap.cc:492] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:43.337957 14326 ts_tablet_manager.cc:1367] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:43.338351 14326 raft_consensus.cc:348] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.338438 14326 raft_consensus.cc:374] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.338466 14326 raft_consensus.cc:729] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.338569 14326 consensus_queue.cc:260] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.338831 14326 ts_tablet_manager.cc:1398] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.345088 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:43.349969 13917 tablet.cc:1722] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:43.350436 14414 tablet.cc:1722] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:43.360683 13918 tablet.cc:1722] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:43.368752 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=d650f95ebfd94e9eb6a5d93819e13074] alter complete (version 3)
I20240426 09:49:43.393399 14524 tablet_bootstrap.cc:492] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:43.393702 14527 tablet_bootstrap.cc:492] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:43.394233 14525 tablet_bootstrap.cc:492] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:43.394642 14043 tablet_bootstrap.cc:492] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:43.394739 14043 ts_tablet_manager.cc:1367] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.064s user 0.000s sys 0.002s
I20240426 09:49:43.395231 14043 raft_consensus.cc:348] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.395345 14043 raft_consensus.cc:374] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.395375 14043 raft_consensus.cc:729] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.395536 14043 consensus_queue.cc:260] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.395833 14043 ts_tablet_manager.cc:1398] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.396253 14529 tablet_bootstrap.cc:492] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:43.397854 14525 tablet_bootstrap.cc:654] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.399035 14527 tablet_bootstrap.cc:654] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.399605 14524 tablet_bootstrap.cc:654] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.400399 14529 tablet_bootstrap.cc:654] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.402274 14524 tablet_bootstrap.cc:492] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:43.402374 14524 ts_tablet_manager.cc:1367] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:43.402915 14524 raft_consensus.cc:348] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:43.403111 14524 raft_consensus.cc:374] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.403216 14524 raft_consensus.cc:729] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.403422 14524 consensus_queue.cc:260] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:43.404349 14524 ts_tablet_manager.cc:1398] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:43.407536 14529 tablet_bootstrap.cc:492] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:43.407701 14529 ts_tablet_manager.cc:1367] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240426 09:49:43.408468 14529 raft_consensus.cc:348] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.408571 14529 raft_consensus.cc:374] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.408605 14529 raft_consensus.cc:729] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.408751 14529 consensus_queue.cc:260] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.409016 14529 ts_tablet_manager.cc:1398] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:43.415871 14525 tablet_bootstrap.cc:492] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:43.415989 14525 ts_tablet_manager.cc:1367] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240426 09:49:43.416455 14525 raft_consensus.cc:348] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:43.416584 14525 raft_consensus.cc:374] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.416615 14525 raft_consensus.cc:729] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.416730 14525 consensus_queue.cc:260] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:43.417104 14525 ts_tablet_manager.cc:1398] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.418998 14527 tablet_bootstrap.cc:492] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:43.419173 14527 ts_tablet_manager.cc:1367] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.001s
I20240426 09:49:43.420444 14527 raft_consensus.cc:348] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.420578 14527 raft_consensus.cc:374] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.420639 14527 raft_consensus.cc:729] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.420768 14527 consensus_queue.cc:260] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.421665 14527 ts_tablet_manager.cc:1398] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240426 09:49:43.421901 14527 tablet_bootstrap.cc:492] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:43.423007 14527 tablet_bootstrap.cc:654] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.424656 14527 tablet_bootstrap.cc:492] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:43.424727 14527 ts_tablet_manager.cc:1367] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:43.425112 14527 raft_consensus.cc:348] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:43.425197 14527 raft_consensus.cc:374] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.425225 14527 raft_consensus.cc:729] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.425323 14527 consensus_queue.cc:260] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:43.425593 14527 ts_tablet_manager.cc:1398] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.425738 14527 tablet_bootstrap.cc:492] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:43.426510 14527 tablet_bootstrap.cc:654] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.428866 14527 tablet_bootstrap.cc:492] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:43.428959 14527 ts_tablet_manager.cc:1367] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:49:43.429421 14527 raft_consensus.cc:348] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.429538 14527 raft_consensus.cc:374] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.429577 14527 raft_consensus.cc:729] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.429698 14527 consensus_queue.cc:260] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.430255 14527 ts_tablet_manager.cc:1398] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:43.434829 14542 ts_tablet_manager.cc:902] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:43.435344 14542 tablet_copy_client.cc:308] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:43.435772 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet a42302cab4524e7a85defe4da40be268 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:40097)
I20240426 09:49:43.435874 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet a42302cab4524e7a85defe4da40be268 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:40097: session id = 3b41d87e7bfb419f8aa7500100129b6c-a42302cab4524e7a85defe4da40be268
I20240426 09:49:43.436796 6902 tablet_copy_source_session.cc:205] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:43.437590 14542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42302cab4524e7a85defe4da40be268. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.439455 14542 tablet_copy_client.cc:783] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:43.439599 14542 tablet_copy_client.cc:647] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:43.440692 14542 tablet_copy_client.cc:523] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:43.441761 14542 tablet_bootstrap.cc:492] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:43.448082 14548 ts_tablet_manager.cc:902] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:43.448666 14548 tablet_copy_client.cc:308] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:43.449117 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet f834d8a08b1341869fd644ff400fceae from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:52457)
I20240426 09:49:43.449229 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet f834d8a08b1341869fd644ff400fceae from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:52457: session id = 3b41d87e7bfb419f8aa7500100129b6c-f834d8a08b1341869fd644ff400fceae
I20240426 09:49:43.450050 7164 tablet_copy_source_session.cc:205] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:43.450907 14548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f834d8a08b1341869fd644ff400fceae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.453006 14548 tablet_copy_client.cc:783] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:43.453138 14548 tablet_copy_client.cc:647] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:43.454682 14548 tablet_copy_client.cc:523] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:43.456184 14548 tablet_bootstrap.cc:492] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:43.467046 14542 tablet_bootstrap.cc:492] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:43.467344 14542 tablet_bootstrap.cc:492] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:43.467509 14542 ts_tablet_manager.cc:1367] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.026s user 0.023s sys 0.002s
I20240426 09:49:43.467928 14542 raft_consensus.cc:348] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.468029 14542 raft_consensus.cc:729] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.468159 14542 consensus_queue.cc:260] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.468755 14542 ts_tablet_manager.cc:1398] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.470934 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-a42302cab4524e7a85defe4da40be268 received from {username='slave'} at 127.0.99.134:40097
I20240426 09:49:43.471019 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-a42302cab4524e7a85defe4da40be268 on tablet a42302cab4524e7a85defe4da40be268 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:43.479025 6734 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:43.479182 6734 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:43.481228 14548 tablet_bootstrap.cc:492] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:43.481529 14548 tablet_bootstrap.cc:492] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:43.481706 14548 ts_tablet_manager.cc:1367] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.026s user 0.022s sys 0.000s
I20240426 09:49:43.482088 6734 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 2a59c860a09f48de97556e93b172b9ff
I20240426 09:49:43.482193 14548 raft_consensus.cc:348] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.482318 14548 raft_consensus.cc:729] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.482496 14548 consensus_queue.cc:260] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.482654 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a59c860a09f48de97556e93b172b9ff with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:43 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:43.483578 14548 ts_tablet_manager.cc:1398] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20240426 09:49:43.484486 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-f834d8a08b1341869fd644ff400fceae received from {username='slave'} at 127.0.99.134:52457
I20240426 09:49:43.484570 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-f834d8a08b1341869fd644ff400fceae on tablet f834d8a08b1341869fd644ff400fceae with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:43.485363 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a59c860a09f48de97556e93b172b9ff with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:43 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:43.486125 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a59c860a09f48de97556e93b172b9ff with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:43 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:43.486788 14585 tablet_replica.cc:330] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:43.486979 14585 raft_consensus.cc:2229] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:43.487213 14585 raft_consensus.cc:2258] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:43.487591 14584 tablet_replica.cc:330] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:43.487823 14584 raft_consensus.cc:2229] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:43.488814 14585 ts_tablet_manager.cc:1875] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:43.489176 14584 raft_consensus.cc:2258] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:43.490447 14584 ts_tablet_manager.cc:1875] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:43.492328 14585 ts_tablet_manager.cc:1888] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:43.492411 14585 log.cc:1196] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/2a59c860a09f48de97556e93b172b9ff
I20240426 09:49:43.492790 14585 ts_tablet_manager.cc:1909] T 2a59c860a09f48de97556e93b172b9ff P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:43.493603 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 2a59c860a09f48de97556e93b172b9ff (table rebalancer_dynamic_table_new_name [id=d650f95ebfd94e9eb6a5d93819e13074]) successfully deleted
I20240426 09:49:43.495182 14586 tablet_replica.cc:330] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:43.495391 14586 raft_consensus.cc:2229] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:43.495612 14586 raft_consensus.cc:2258] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:43.496213 14586 ts_tablet_manager.cc:1875] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:43.496614 14584 ts_tablet_manager.cc:1888] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:43.496794 14584 log.cc:1196] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/2a59c860a09f48de97556e93b172b9ff
I20240426 09:49:43.499769 14586 ts_tablet_manager.cc:1888] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:43.499835 14586 log.cc:1196] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/2a59c860a09f48de97556e93b172b9ff
I20240426 09:49:43.500088 14586 ts_tablet_manager.cc:1909] T 2a59c860a09f48de97556e93b172b9ff P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:43.497174 14584 ts_tablet_manager.cc:1909] T 2a59c860a09f48de97556e93b172b9ff P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:43.500715 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 2a59c860a09f48de97556e93b172b9ff (table rebalancer_dynamic_table_new_name [id=d650f95ebfd94e9eb6a5d93819e13074]) successfully deleted
I20240426 09:49:43.501102 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 2a59c860a09f48de97556e93b172b9ff (table rebalancer_dynamic_table_new_name [id=d650f95ebfd94e9eb6a5d93819e13074]) successfully deleted
I20240426 09:49:43.519170 13569 raft_consensus.cc:482] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:43.519297 13569 raft_consensus.cc:504] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:43.519678 13569 leader_election.cc:288] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:43.520082 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d49693f3ade40f2a9d5a0029dd7508b" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:43.520237 7142 raft_consensus.cc:2454] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:43.520289 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d49693f3ade40f2a9d5a0029dd7508b" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:43.520426 7013 raft_consensus.cc:2454] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:43.520623 9350 leader_election.cc:302] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:43.520946 13569 raft_consensus.cc:2781] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:43.520999 13569 raft_consensus.cc:482] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:43.521034 13569 raft_consensus.cc:3037] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.522073 13569 raft_consensus.cc:504] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:43.522394 13569 leader_election.cc:288] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:43.522984 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d49693f3ade40f2a9d5a0029dd7508b" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:43.523108 7013 raft_consensus.cc:3037] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.524121 7013 raft_consensus.cc:2454] T 4d49693f3ade40f2a9d5a0029dd7508b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:43.524348 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d49693f3ade40f2a9d5a0029dd7508b" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:43.524417 9350 leader_election.cc:302] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:43.524456 7142 raft_consensus.cc:3037] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.524619 13569 raft_consensus.cc:2781] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:43.524714 13569 raft_consensus.cc:686] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:43.524837 13569 consensus_queue.cc:237] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:43.525386 7142 raft_consensus.cc:2454] T 4d49693f3ade40f2a9d5a0029dd7508b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:43.527273 6737 catalog_manager.cc:5484] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:43.567909 14629 ts_tablet_manager.cc:902] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:43.568452 14629 tablet_copy_client.cc:308] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:43.568940 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 4fb0f5d2d55a45599c1b09c5684522e6 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:41455)
I20240426 09:49:43.569052 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 4fb0f5d2d55a45599c1b09c5684522e6 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:41455: session id = 558c4ecd0ece460d9591aa7a3c799d19-4fb0f5d2d55a45599c1b09c5684522e6
I20240426 09:49:43.570135 7295 tablet_copy_source_session.cc:205] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:43.575397 14629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb0f5d2d55a45599c1b09c5684522e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.578012 14629 tablet_copy_client.cc:783] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:43.578253 14629 tablet_copy_client.cc:647] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:43.579435 14629 tablet_copy_client.cc:523] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:43.580684 14629 tablet_bootstrap.cc:492] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:43.595695 6736 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:43.601927 9395 tablet_service.cc:1467] Processing CreateTablet for tablet e51f6248665b405eb1b211cbcb75233c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a9dc0a42250f4730a092920287ca9acf]), partition=
I20240426 09:49:43.602010 9527 tablet_service.cc:1467] Processing CreateTablet for tablet e51f6248665b405eb1b211cbcb75233c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a9dc0a42250f4730a092920287ca9acf]), partition=
I20240426 09:49:43.602213 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51f6248665b405eb1b211cbcb75233c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.604710 14527 tablet_bootstrap.cc:492] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:43.604948 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51f6248665b405eb1b211cbcb75233c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.605631 14527 tablet_bootstrap.cc:654] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.607791 14043 tablet_bootstrap.cc:492] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:43.608713 14043 tablet_bootstrap.cc:654] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.609076 9266 tablet_service.cc:1467] Processing CreateTablet for tablet e51f6248665b405eb1b211cbcb75233c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a9dc0a42250f4730a092920287ca9acf]), partition=
I20240426 09:49:43.609299 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51f6248665b405eb1b211cbcb75233c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.610311 14043 tablet_bootstrap.cc:492] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:43.610394 14043 ts_tablet_manager.cc:1367] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:49:43.610986 14043 raft_consensus.cc:348] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.611127 14043 raft_consensus.cc:374] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.611168 14043 raft_consensus.cc:729] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.611308 14043 consensus_queue.cc:260] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.611645 14043 ts_tablet_manager.cc:1398] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.612433 14326 tablet_bootstrap.cc:492] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:43.613484 14326 tablet_bootstrap.cc:654] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:43.621186 14629 tablet_bootstrap.cc:492] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:43.621632 14629 tablet_bootstrap.cc:492] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:43.621884 14629 ts_tablet_manager.cc:1367] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.041s user 0.021s sys 0.007s
I20240426 09:49:43.622112 14326 tablet_bootstrap.cc:492] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:43.622217 14326 ts_tablet_manager.cc:1367] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:43.622334 14629 raft_consensus.cc:348] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:43.622431 14629 raft_consensus.cc:729] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.622542 14629 consensus_queue.cc:260] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:43.622690 14527 tablet_bootstrap.cc:492] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:43.622810 14527 ts_tablet_manager.cc:1367] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.001s
I20240426 09:49:43.622736 14326 raft_consensus.cc:348] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.622861 14326 raft_consensus.cc:374] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.622910 14629 ts_tablet_manager.cc:1398] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.622911 14326 raft_consensus.cc:729] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.623028 14326 consensus_queue.cc:260] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.623349 14326 ts_tablet_manager.cc:1398] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.623333 14527 raft_consensus.cc:348] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.623452 14527 raft_consensus.cc:374] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:43.623484 14527 raft_consensus.cc:729] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.623590 14527 consensus_queue.cc:260] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.623847 14527 ts_tablet_manager.cc:1398] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.624034 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-4fb0f5d2d55a45599c1b09c5684522e6 received from {username='slave'} at 127.0.99.136:41455
I20240426 09:49:43.624117 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-4fb0f5d2d55a45599c1b09c5684522e6 on tablet 4fb0f5d2d55a45599c1b09c5684522e6 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:43.629215 14638 ts_tablet_manager.cc:902] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:43.629818 14638 tablet_copy_client.cc:308] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:43.630277 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet bbdcd6105a2647738257638e1e1096c3 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:46939)
I20240426 09:49:43.630389 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet bbdcd6105a2647738257638e1e1096c3 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:46939: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-bbdcd6105a2647738257638e1e1096c3
I20240426 09:49:43.631680 9685 tablet_copy_source_session.cc:205] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:43.632606 14638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdcd6105a2647738257638e1e1096c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.633911 14474 raft_consensus.cc:482] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:43.634009 14474 raft_consensus.cc:504] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.634403 14474 leader_election.cc:288] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:43.635084 14638 tablet_copy_client.cc:783] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:43.635242 14638 tablet_copy_client.cc:647] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:43.635375 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d91861708146eb94dcc2cdef1517eb" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:43.635505 9416 raft_consensus.cc:2454] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:43.635883 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d91861708146eb94dcc2cdef1517eb" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:43.636008 9548 raft_consensus.cc:2454] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:43.636168 9219 leader_election.cc:302] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:43.636337 14474 raft_consensus.cc:2781] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:43.636416 14474 raft_consensus.cc:482] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:43.636453 14474 raft_consensus.cc:3037] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.637501 14638 tablet_copy_client.cc:523] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:43.637459 14474 raft_consensus.cc:504] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.637782 14474 leader_election.cc:288] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:43.638211 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d91861708146eb94dcc2cdef1517eb" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:43.638367 9416 raft_consensus.cc:3037] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.638342 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d91861708146eb94dcc2cdef1517eb" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:43.638449 9549 raft_consensus.cc:3037] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.639230 9416 raft_consensus.cc:2454] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:43.639413 9549 raft_consensus.cc:2454] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:43.639672 9219 leader_election.cc:302] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:43.639833 14638 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:43.639876 12388 raft_consensus.cc:2781] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:43.639946 12388 raft_consensus.cc:686] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:43.640064 12388 consensus_queue.cc:237] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.643741 6736 catalog_manager.cc:5484] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:43.643889 14638 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:43.644121 14638 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:43.644274 14638 ts_tablet_manager.cc:1367] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20240426 09:49:43.644722 14638 raft_consensus.cc:348] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:43.644819 14638 raft_consensus.cc:729] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.644934 14638 consensus_queue.cc:260] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:43.646260 14638 ts_tablet_manager.cc:1398] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:43.646723 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-bbdcd6105a2647738257638e1e1096c3 received from {username='slave'} at 127.0.99.135:46939
I20240426 09:49:43.646826 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-bbdcd6105a2647738257638e1e1096c3 on tablet bbdcd6105a2647738257638e1e1096c3 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:43.669430 14548 ts_tablet_manager.cc:902] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:43.670075 14548 tablet_copy_client.cc:308] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:43.670655 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet e856ff2c8ffb4bd4b44c694047c8a8fa from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:46185)
I20240426 09:49:43.670786 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet e856ff2c8ffb4bd4b44c694047c8a8fa from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:46185: session id = 3b41d87e7bfb419f8aa7500100129b6c-e856ff2c8ffb4bd4b44c694047c8a8fa
I20240426 09:49:43.671618 9685 tablet_copy_source_session.cc:205] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:43.672410 14548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e856ff2c8ffb4bd4b44c694047c8a8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.674392 14548 tablet_copy_client.cc:783] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:43.674572 14548 tablet_copy_client.cc:647] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:43.675750 14548 tablet_copy_client.cc:523] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:43.675905 14638 ts_tablet_manager.cc:902] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:43.676329 14638 tablet_copy_client.cc:308] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:43.676823 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 639d1d6189fb4d5f8a2d0aa0d39575ec from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:49245)
I20240426 09:49:43.676929 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 639d1d6189fb4d5f8a2d0aa0d39575ec from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:49245: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-639d1d6189fb4d5f8a2d0aa0d39575ec
I20240426 09:49:43.676985 14548 tablet_bootstrap.cc:492] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:43.677657 7164 tablet_copy_source_session.cc:205] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:43.678464 14638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 639d1d6189fb4d5f8a2d0aa0d39575ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.680264 14638 tablet_copy_client.cc:783] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:43.680445 14638 tablet_copy_client.cc:647] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:43.680608 14548 tablet_bootstrap.cc:492] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:43.680964 14548 tablet_bootstrap.cc:492] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:43.681102 14548 ts_tablet_manager.cc:1367] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20240426 09:49:43.681540 14548 raft_consensus.cc:348] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.681645 14548 raft_consensus.cc:729] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.681749 14638 tablet_copy_client.cc:523] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:43.681775 14548 consensus_queue.cc:260] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.683694 14638 tablet_bootstrap.cc:492] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:43.685298 14548 ts_tablet_manager.cc:1398] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:43.687516 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-e856ff2c8ffb4bd4b44c694047c8a8fa received from {username='slave'} at 127.0.99.134:46185
I20240426 09:49:43.687602 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-e856ff2c8ffb4bd4b44c694047c8a8fa on tablet e856ff2c8ffb4bd4b44c694047c8a8fa with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:43.697695 14655 ts_tablet_manager.cc:902] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:43.698997 14655 tablet_copy_client.cc:308] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:43.699455 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 852fad44c59f42dd9ce66b84819a9296 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:57967)
I20240426 09:49:43.699576 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 852fad44c59f42dd9ce66b84819a9296 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:57967: session id = a692602e1a2e4794877e6fb92a7b9962-852fad44c59f42dd9ce66b84819a9296
I20240426 09:49:43.700578 7033 tablet_copy_source_session.cc:205] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:43.701838 14655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 852fad44c59f42dd9ce66b84819a9296. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.704131 14655 tablet_copy_client.cc:783] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:43.704797 14655 tablet_copy_client.cc:647] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:43.707298 14655 tablet_copy_client.cc:523] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:43.709528 14655 tablet_bootstrap.cc:492] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:43.710100 14638 tablet_bootstrap.cc:492] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:43.710341 14638 tablet_bootstrap.cc:492] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:43.710489 14638 ts_tablet_manager.cc:1367] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.027s user 0.019s sys 0.002s
I20240426 09:49:43.710965 14638 raft_consensus.cc:348] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:43.711078 14638 raft_consensus.cc:729] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.711205 14638 consensus_queue.cc:260] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:43.711529 14638 ts_tablet_manager.cc:1398] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240426 09:49:43.712867 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-639d1d6189fb4d5f8a2d0aa0d39575ec received from {username='slave'} at 127.0.99.135:49245
I20240426 09:49:43.712941 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-639d1d6189fb4d5f8a2d0aa0d39575ec on tablet 639d1d6189fb4d5f8a2d0aa0d39575ec with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:43.717648 14655 tablet_bootstrap.cc:492] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:43.717898 14655 tablet_bootstrap.cc:492] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:43.718036 14655 ts_tablet_manager.cc:1367] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20240426 09:49:43.718425 14655 raft_consensus.cc:348] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.718556 14655 raft_consensus.cc:729] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.718708 14655 consensus_queue.cc:260] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.719029 14655 ts_tablet_manager.cc:1398] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:43.719523 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-852fad44c59f42dd9ce66b84819a9296 received from {username='slave'} at 127.0.99.133:57967
I20240426 09:49:43.719620 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-852fad44c59f42dd9ce66b84819a9296 on tablet 852fad44c59f42dd9ce66b84819a9296 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:43.734953 14629 ts_tablet_manager.cc:902] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:43.735416 14629 tablet_copy_client.cc:308] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:43.736061 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 643fd2d359a14f8eaa8d8a0d0879ef61 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:41455)
I20240426 09:49:43.736174 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 643fd2d359a14f8eaa8d8a0d0879ef61 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:41455: session id = 558c4ecd0ece460d9591aa7a3c799d19-643fd2d359a14f8eaa8d8a0d0879ef61
I20240426 09:49:43.737103 7295 tablet_copy_source_session.cc:205] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:43.737993 14629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 643fd2d359a14f8eaa8d8a0d0879ef61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:43.740176 14629 tablet_copy_client.cc:783] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:43.740330 14629 tablet_copy_client.cc:647] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:43.741726 14629 tablet_copy_client.cc:523] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:43.743182 14629 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:43.763851 13916 raft_consensus.cc:482] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:43.763996 13916 raft_consensus.cc:504] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.764438 13916 leader_election.cc:288] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:43.764866 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aebef9b26fec446b8e71d4046768aedc" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:43.765013 9416 raft_consensus.cc:2454] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:43.765357 9219 leader_election.cc:302] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:43.766172 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aebef9b26fec446b8e71d4046768aedc" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:43.766316 9672 raft_consensus.cc:2454] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:43.766484 14629 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:43.766656 12388 raft_consensus.cc:2781] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:43.766726 12388 raft_consensus.cc:482] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:43.766765 12388 raft_consensus.cc:3037] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.766757 14629 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:43.766963 14629 ts_tablet_manager.cc:1367] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.024s user 0.019s sys 0.002s
I20240426 09:49:43.767418 14629 raft_consensus.cc:348] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:43.767781 14629 raft_consensus.cc:729] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:43.767796 12388 raft_consensus.cc:504] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.767935 14629 consensus_queue.cc:260] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:43.768183 12388 leader_election.cc:288] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:43.768507 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aebef9b26fec446b8e71d4046768aedc" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:43.768613 9672 raft_consensus.cc:3037] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.768877 14629 ts_tablet_manager.cc:1398] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:43.769321 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-643fd2d359a14f8eaa8d8a0d0879ef61 received from {username='slave'} at 127.0.99.136:41455
I20240426 09:49:43.769407 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-643fd2d359a14f8eaa8d8a0d0879ef61 on tablet 643fd2d359a14f8eaa8d8a0d0879ef61 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:43.769568 9672 raft_consensus.cc:2454] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:43.769953 9219 leader_election.cc:302] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:43.770001 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aebef9b26fec446b8e71d4046768aedc" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:43.770108 12388 raft_consensus.cc:2781] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:43.770150 9416 raft_consensus.cc:3037] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.770190 12388 raft_consensus.cc:686] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:43.770319 12388 consensus_queue.cc:237] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:43.771137 9416 raft_consensus.cc:2454] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:43.773497 6731 catalog_manager.cc:5484] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:43.790894 9395 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:43.792132 6860 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:43.798736 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:43.805667 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:43.810926 9266 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:43.814951 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:43.815264 9660 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:43.815913 6993 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:43.832531 12476 raft_consensus.cc:482] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:43.832681 12476 raft_consensus.cc:504] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.833130 12476 leader_election.cc:288] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:43.835372 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51f6248665b405eb1b211cbcb75233c" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:43.835541 9416 raft_consensus.cc:2454] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:43.835811 9482 leader_election.cc:302] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:43.835983 12476 raft_consensus.cc:2781] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:43.836031 12476 raft_consensus.cc:482] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:43.836062 12476 raft_consensus.cc:3037] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.837108 12476 raft_consensus.cc:504] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.837464 12476 leader_election.cc:288] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:43.837867 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51f6248665b405eb1b211cbcb75233c" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:43.838002 9285 raft_consensus.cc:3037] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.838083 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51f6248665b405eb1b211cbcb75233c" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:43.838186 9416 raft_consensus.cc:3037] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:43.839008 9285 raft_consensus.cc:2454] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:43.839131 9416 raft_consensus.cc:2454] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:43.839737 9483 leader_election.cc:302] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:43.840317 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51f6248665b405eb1b211cbcb75233c" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:43.840469 9286 raft_consensus.cc:2362] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1. Re-sending same reply.
I20240426 09:49:43.843058 12476 raft_consensus.cc:2781] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:43.843218 12476 raft_consensus.cc:686] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:43.843372 12476 consensus_queue.cc:237] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:43.845825 6731 catalog_manager.cc:5484] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:43.860780 9416 raft_consensus.cc:1203] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:43.957317 13569 consensus_queue.cc:1035] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.961686 14522 consensus_queue.cc:1035] T 4d49693f3ade40f2a9d5a0029dd7508b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.983055 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:43.984282 6993 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:43.989729 9395 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:43.991462 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:43.994983 9527 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:43.995443 6860 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:43.997156 9286 raft_consensus.cc:1261] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:43.997582 9266 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:43.997565 14678 consensus_queue.cc:1035] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:43.998095 9415 raft_consensus.cc:1261] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:43.998317 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:44.000452 14678 consensus_queue.cc:1035] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.000986 6739 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet e51f6248665b405eb1b211cbcb75233c (table rebalancer_dynamic_table [id=a9dc0a42250f4730a092920287ca9acf]). Expected version 1 got 0
I20240426 09:49:44.002274 9415 raft_consensus.cc:1203] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:44.003554 6739 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet e51f6248665b405eb1b211cbcb75233c (table rebalancer_dynamic_table [id=a9dc0a42250f4730a092920287ca9acf]). Expected version 1 got 0
I20240426 09:49:44.009893 14743 tablet.cc:1722] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:44.011495 14743 tablet.cc:1620] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:44.011811 6739 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet e51f6248665b405eb1b211cbcb75233c (table rebalancer_dynamic_table [id=a9dc0a42250f4730a092920287ca9acf]). Expected version 1 got 0
I20240426 09:49:44.015211 6739 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet e51f6248665b405eb1b211cbcb75233c (table rebalancer_dynamic_table [id=a9dc0a42250f4730a092920287ca9acf]). Expected version 1 got 0
I20240426 09:49:44.016681 14742 tablet.cc:1722] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:44.018268 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=a9dc0a42250f4730a092920287ca9acf] alter complete (version 1)
I20240426 09:49:44.022362 14741 tablet.cc:1722] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:44.022718 14741 tablet.cc:1620] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:44.023083 14742 tablet.cc:1620] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:44.024277 9548 raft_consensus.cc:1203] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:44.039552 9660 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:44.076491 9286 raft_consensus.cc:1203] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:44.127120 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:44.133226 14741 tablet.cc:1722] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:44.133602 14741 tablet.cc:1620] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:44.133922 14742 tablet.cc:1722] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:44.134210 14742 tablet.cc:1620] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:44.139600 14743 tablet.cc:1722] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:44.139954 14743 tablet.cc:1620] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:44.146698 6740 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=a9dc0a42250f4730a092920287ca9acf] alter complete (version 2)
I20240426 09:49:44.148367 9548 raft_consensus.cc:1203] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:44.150549 9672 raft_consensus.cc:1203] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:44.178787 9415 raft_consensus.cc:1203] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:44.187188 9672 raft_consensus.cc:1203] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:44.197623 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:44.199929 6993 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:44.201828 12388 consensus_queue.cc:1035] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.201938 9266 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:44.207095 9395 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:44.209908 13916 consensus_queue.cc:1035] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.219988 9660 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:44.220275 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:44.230455 9527 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:44.235507 12388 consensus_queue.cc:1035] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.235782 12388 consensus_queue.cc:1035] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.243185 6860 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:44.259428 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:44.266507 14741 tablet.cc:1722] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:44.267870 14742 tablet.cc:1722] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:44.269237 14743 tablet.cc:1722] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:44.274238 6740 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=a9dc0a42250f4730a092920287ca9acf] alter complete (version 3)
I20240426 09:49:44.314182 7275 consensus_queue.cc:237] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.315485 9672 raft_consensus.cc:1261] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:44.315485 7011 raft_consensus.cc:1261] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:44.315891 14398 consensus_queue.cc:1035] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:44.316011 14796 consensus_queue.cc:1035] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:44.317796 14822 raft_consensus.cc:2932] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:44.318255 7011 raft_consensus.cc:2932] T 4fb0f5d2d55a45599c1b09c5684522e6 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:44.318334 9672 raft_consensus.cc:2932] T 4fb0f5d2d55a45599c1b09c5684522e6 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:44.321754 6740 catalog_manager.cc:5484] T 4fb0f5d2d55a45599c1b09c5684522e6 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:44.327587 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fb0f5d2d55a45599c1b09c5684522e6 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:44.327975 14828 tablet_replica.cc:330] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:44.328560 14828 raft_consensus.cc:2229] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:44.329373 14828 raft_consensus.cc:2258] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:44.329959 14828 ts_tablet_manager.cc:1875] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:44.332605 14828 ts_tablet_manager.cc:1888] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:44.332681 14828 log.cc:1196] T 4fb0f5d2d55a45599c1b09c5684522e6 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/4fb0f5d2d55a45599c1b09c5684522e6
I20240426 09:49:44.333238 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 4fb0f5d2d55a45599c1b09c5684522e6 (table rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]) successfully deleted
I20240426 09:49:44.340919 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:44.348659 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 360aa06557974c5ea635a32245c0903e (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:44.348903 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 360aa06557974c5ea635a32245c0903e (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:44.349035 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360aa06557974c5ea635a32245c0903e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.349148 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360aa06557974c5ea635a32245c0903e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.351642 9395 tablet_service.cc:1467] Processing CreateTablet for tablet 538173a16ff94e70bc8d25b8f1e7c84e (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:44.351924 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538173a16ff94e70bc8d25b8f1e7c84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.352046 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8803aa38a646ebbdf5018ab7f65845 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:44.352293 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8803aa38a646ebbdf5018ab7f65845. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.354494 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 538173a16ff94e70bc8d25b8f1e7c84e (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:44.354754 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538173a16ff94e70bc8d25b8f1e7c84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.356264 14836 tablet_bootstrap.cc:492] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:44.356953 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 360aa06557974c5ea635a32245c0903e (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:44.357228 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360aa06557974c5ea635a32245c0903e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.357229 14836 tablet_bootstrap.cc:654] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.357843 14839 tablet_bootstrap.cc:492] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:44.358845 14839 tablet_bootstrap.cc:654] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.358990 14837 tablet_bootstrap.cc:492] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:44.359688 14847 tablet_bootstrap.cc:492] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:44.360163 14836 tablet_bootstrap.cc:492] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:44.360265 14836 ts_tablet_manager.cc:1367] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:44.360592 14847 tablet_bootstrap.cc:654] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.360796 14836 raft_consensus.cc:348] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.360917 14836 raft_consensus.cc:374] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.360953 14836 raft_consensus.cc:729] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.361608 14837 tablet_bootstrap.cc:654] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.362360 14847 tablet_bootstrap.cc:492] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:44.362452 14847 ts_tablet_manager.cc:1367] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:44.362100 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8803aa38a646ebbdf5018ab7f65845 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:44.362859 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8803aa38a646ebbdf5018ab7f65845. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.362846 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8803aa38a646ebbdf5018ab7f65845 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:44.362908 14847 raft_consensus.cc:348] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.363001 14847 raft_consensus.cc:374] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.363034 14847 raft_consensus.cc:729] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.363391 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 538173a16ff94e70bc8d25b8f1e7c84e (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:44.363622 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538173a16ff94e70bc8d25b8f1e7c84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.363091 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8803aa38a646ebbdf5018ab7f65845. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.365723 14836 consensus_queue.cc:260] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.366132 14836 ts_tablet_manager.cc:1398] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:44.366438 14836 tablet_bootstrap.cc:492] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:44.363159 14847 consensus_queue.cc:260] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.366874 14847 ts_tablet_manager.cc:1398] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:44.368332 14835 raft_consensus.cc:482] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:44.368419 14835 raft_consensus.cc:504] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.368834 14835 leader_election.cc:288] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:44.369454 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360aa06557974c5ea635a32245c0903e" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:44.369863 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360aa06557974c5ea635a32245c0903e" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:44.369994 9549 raft_consensus.cc:2454] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
W20240426 09:49:44.370311 9614 leader_election.cc:341] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:44.371013 9614 leader_election.cc:302] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters: 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:44.371325 14835 raft_consensus.cc:2781] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:44.371394 14835 raft_consensus.cc:482] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:44.371454 14835 raft_consensus.cc:3037] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.372730 14835 raft_consensus.cc:504] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.373064 14835 leader_election.cc:288] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:44.373483 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360aa06557974c5ea635a32245c0903e" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:44.373507 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360aa06557974c5ea635a32245c0903e" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:44.373590 9549 raft_consensus.cc:3037] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
W20240426 09:49:44.374043 9614 leader_election.cc:341] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:44.374516 9549 raft_consensus.cc:2454] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:44.374913 9614 leader_election.cc:302] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters: 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:44.375794 14361 raft_consensus.cc:2781] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:44.375970 14361 raft_consensus.cc:686] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:44.376104 14361 consensus_queue.cc:237] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.376300 6731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:44.376425 6731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:44.376610 14836 tablet_bootstrap.cc:654] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.377380 14838 tablet_bootstrap.cc:492] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:44.378626 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet e51f6248665b405eb1b211cbcb75233c
I20240426 09:49:44.379607 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet e51f6248665b405eb1b211cbcb75233c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:44 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:44.380865 6731 catalog_manager.cc:5484] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:44.380980 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet e51f6248665b405eb1b211cbcb75233c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:44 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:44.381098 9395 tablet_service.cc:1514] Processing DeleteTablet for tablet e51f6248665b405eb1b211cbcb75233c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:44 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:44.381407 14859 tablet_replica.cc:330] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:44.381630 14859 raft_consensus.cc:2229] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:44.381722 14838 tablet_bootstrap.cc:654] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.381793 14859 raft_consensus.cc:2258] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:44.382066 14847 tablet_bootstrap.cc:492] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:44.382990 14847 tablet_bootstrap.cc:654] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.383740 14853 tablet_replica.cc:330] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:44.383937 14853 raft_consensus.cc:2229] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:44.384136 14853 raft_consensus.cc:2258] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:44.384433 14859 ts_tablet_manager.cc:1875] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:44.384864 14853 ts_tablet_manager.cc:1875] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:44.387019 14853 ts_tablet_manager.cc:1888] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:44.387131 14853 log.cc:1196] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/e51f6248665b405eb1b211cbcb75233c
I20240426 09:49:44.387365 14837 tablet_bootstrap.cc:492] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:44.387382 14838 tablet_bootstrap.cc:492] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:44.387475 14838 ts_tablet_manager.cc:1367] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:44.387544 14837 ts_tablet_manager.cc:1367] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20240426 09:49:44.387697 14859 ts_tablet_manager.cc:1888] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:44.387777 14859 log.cc:1196] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/e51f6248665b405eb1b211cbcb75233c
I20240426 09:49:44.387887 14853 ts_tablet_manager.cc:1909] T e51f6248665b405eb1b211cbcb75233c P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:44.388051 14859 ts_tablet_manager.cc:1909] T e51f6248665b405eb1b211cbcb75233c P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:44.388263 14837 raft_consensus.cc:348] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.388449 14837 raft_consensus.cc:374] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.388595 14837 raft_consensus.cc:729] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.388634 14838 raft_consensus.cc:348] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.388849 14838 raft_consensus.cc:374] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.388931 14838 raft_consensus.cc:729] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.388813 14837 consensus_queue.cc:260] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.389070 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet e51f6248665b405eb1b211cbcb75233c (table rebalancer_dynamic_table_new_name [id=a9dc0a42250f4730a092920287ca9acf]) successfully deleted
I20240426 09:49:44.389264 14837 ts_tablet_manager.cc:1398] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:44.389928 14847 tablet_bootstrap.cc:492] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:44.390034 14847 ts_tablet_manager.cc:1367] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:44.390077 14836 tablet_bootstrap.cc:492] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:44.390172 14836 ts_tablet_manager.cc:1367] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20240426 09:49:44.390403 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet e51f6248665b405eb1b211cbcb75233c (table rebalancer_dynamic_table_new_name [id=a9dc0a42250f4730a092920287ca9acf]) successfully deleted
I20240426 09:49:44.390506 14847 raft_consensus.cc:348] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.390604 14847 raft_consensus.cc:374] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.390559 14838 consensus_queue.cc:260] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.390635 14847 raft_consensus.cc:729] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.390584 14836 raft_consensus.cc:348] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.390681 14836 raft_consensus.cc:374] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.390709 14836 raft_consensus.cc:729] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.390864 14836 consensus_queue.cc:260] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.391147 14836 ts_tablet_manager.cc:1398] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:44.390748 14847 consensus_queue.cc:260] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.391315 14836 tablet_bootstrap.cc:492] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:44.391422 14847 ts_tablet_manager.cc:1398] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:44.392269 14836 tablet_bootstrap.cc:654] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.393314 14838 ts_tablet_manager.cc:1398] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:44.396729 14858 tablet_replica.cc:330] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:44.396927 14858 raft_consensus.cc:2229] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:44.397755 14858 raft_consensus.cc:2258] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:44.398183 14836 tablet_bootstrap.cc:492] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:44.398295 14836 ts_tablet_manager.cc:1367] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:44.398739 14836 raft_consensus.cc:348] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.398906 14836 raft_consensus.cc:374] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.398949 14836 raft_consensus.cc:729] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.399078 14836 consensus_queue.cc:260] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.399454 14836 ts_tablet_manager.cc:1398] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:44.402338 14839 tablet_bootstrap.cc:492] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:44.402463 14839 ts_tablet_manager.cc:1367] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.045s user 0.000s sys 0.002s
I20240426 09:49:44.402966 14839 raft_consensus.cc:348] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.403095 14839 raft_consensus.cc:374] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.403131 14839 raft_consensus.cc:729] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.403239 14839 consensus_queue.cc:260] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.403512 14839 ts_tablet_manager.cc:1398] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:44.403656 14839 tablet_bootstrap.cc:492] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:44.404745 14839 tablet_bootstrap.cc:654] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.405861 14839 tablet_bootstrap.cc:492] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:44.405937 14839 ts_tablet_manager.cc:1367] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240426 09:49:44.406332 14839 raft_consensus.cc:348] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.406433 14839 raft_consensus.cc:374] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.406461 14839 raft_consensus.cc:729] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.406641 14839 consensus_queue.cc:260] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.407307 14858 ts_tablet_manager.cc:1875] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:44.407377 14839 ts_tablet_manager.cc:1398] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:44.409436 14858 ts_tablet_manager.cc:1888] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:44.409503 14858 log.cc:1196] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/e51f6248665b405eb1b211cbcb75233c
I20240426 09:49:44.409773 14858 ts_tablet_manager.cc:1909] T e51f6248665b405eb1b211cbcb75233c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:44.410410 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet e51f6248665b405eb1b211cbcb75233c (table rebalancer_dynamic_table_new_name [id=a9dc0a42250f4730a092920287ca9acf]) successfully deleted
I20240426 09:49:44.429711 14319 raft_consensus.cc:482] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:44.429870 14319 raft_consensus.cc:504] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.430370 14319 leader_election.cc:288] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:44.431277 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538173a16ff94e70bc8d25b8f1e7c84e" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:44.431421 9415 raft_consensus.cc:2454] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:44.431684 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538173a16ff94e70bc8d25b8f1e7c84e" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:44.431804 9549 raft_consensus.cc:2454] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:44.432773 6815 leader_election.cc:302] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:44.434129 14319 raft_consensus.cc:2781] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:44.434198 14319 raft_consensus.cc:482] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:44.434237 14319 raft_consensus.cc:3037] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.435524 14319 raft_consensus.cc:504] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.435942 14319 leader_election.cc:288] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:44.436853 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538173a16ff94e70bc8d25b8f1e7c84e" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:44.436980 9415 raft_consensus.cc:3037] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.438061 9415 raft_consensus.cc:2454] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:44.438663 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538173a16ff94e70bc8d25b8f1e7c84e" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:44.438807 9549 raft_consensus.cc:3037] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.439797 9549 raft_consensus.cc:2454] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:44.440567 6815 leader_election.cc:302] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:44.440953 14319 raft_consensus.cc:2781] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:44.441130 14319 raft_consensus.cc:686] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:44.441269 14319 consensus_queue.cc:237] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.443495 6734 catalog_manager.cc:5484] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:44.477133 14885 raft_consensus.cc:482] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:44.477250 14885 raft_consensus.cc:504] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.477567 14885 leader_election.cc:288] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:44.477934 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f8803aa38a646ebbdf5018ab7f65845" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:44.478027 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f8803aa38a646ebbdf5018ab7f65845" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:44.478097 9672 raft_consensus.cc:2454] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:44.478152 9549 raft_consensus.cc:2454] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:44.478497 6815 leader_election.cc:302] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:44.478744 14885 raft_consensus.cc:2781] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:44.478821 14885 raft_consensus.cc:482] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:44.478869 14885 raft_consensus.cc:3037] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.480203 14885 raft_consensus.cc:504] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.480355 6731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:44.480868 14885 leader_election.cc:288] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:44.481211 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f8803aa38a646ebbdf5018ab7f65845" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:44.481333 9672 raft_consensus.cc:3037] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.481323 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f8803aa38a646ebbdf5018ab7f65845" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:44.481451 9549 raft_consensus.cc:3037] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.482393 9672 raft_consensus.cc:2454] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:44.482398 9549 raft_consensus.cc:2454] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:44.482733 6815 leader_election.cc:302] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:44.482906 14885 raft_consensus.cc:2781] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:44.483023 14885 raft_consensus.cc:686] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:44.483206 14885 consensus_queue.cc:237] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:44.486066 6731 catalog_manager.cc:5484] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:44.486519 7122 tablet_service.cc:1467] Processing CreateTablet for tablet fa260dfdb2e241e795368e69e1d3b1b5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d1c4d6597cfe43cc8cd4f0358381f9aa]), partition=
I20240426 09:49:44.486805 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa260dfdb2e241e795368e69e1d3b1b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.488927 14837 tablet_bootstrap.cc:492] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:44.489856 14837 tablet_bootstrap.cc:654] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.489902 9660 tablet_service.cc:1467] Processing CreateTablet for tablet fa260dfdb2e241e795368e69e1d3b1b5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d1c4d6597cfe43cc8cd4f0358381f9aa]), partition=
I20240426 09:49:44.490121 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa260dfdb2e241e795368e69e1d3b1b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.490242 9395 tablet_service.cc:1467] Processing CreateTablet for tablet fa260dfdb2e241e795368e69e1d3b1b5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d1c4d6597cfe43cc8cd4f0358381f9aa]), partition=
I20240426 09:49:44.490463 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa260dfdb2e241e795368e69e1d3b1b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:44.490895 14837 tablet_bootstrap.cc:492] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:44.490957 14837 ts_tablet_manager.cc:1367] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:44.491281 14837 raft_consensus.cc:348] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.491350 14837 raft_consensus.cc:374] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.491372 14837 raft_consensus.cc:729] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.491508 14837 consensus_queue.cc:260] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.491752 14837 ts_tablet_manager.cc:1398] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:44.492267 14847 tablet_bootstrap.cc:492] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:44.492559 14838 tablet_bootstrap.cc:492] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:44.493160 14847 tablet_bootstrap.cc:654] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.493575 14838 tablet_bootstrap.cc:654] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:44.494370 14847 tablet_bootstrap.cc:492] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:44.494455 14847 ts_tablet_manager.cc:1367] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240426 09:49:44.494946 14847 raft_consensus.cc:348] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.495090 14847 raft_consensus.cc:374] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.495121 14847 raft_consensus.cc:729] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.495258 14847 consensus_queue.cc:260] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.498006 14847 ts_tablet_manager.cc:1398] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:44.499413 14838 tablet_bootstrap.cc:492] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:44.499516 14838 ts_tablet_manager.cc:1367] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20240426 09:49:44.501346 14838 raft_consensus.cc:348] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.501508 14838 raft_consensus.cc:374] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:44.501554 14838 raft_consensus.cc:729] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:44.501679 14838 consensus_queue.cc:260] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.501976 14838 ts_tablet_manager.cc:1398] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:44.510957 6993 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:44.511884 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:44.518371 9395 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:44.531253 6860 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:44.531610 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:44.540715 9660 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:44.542444 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:44.544381 9266 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:44.630290 14850 raft_consensus.cc:482] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:44.630398 14850 raft_consensus.cc:504] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.630844 14850 leader_election.cc:288] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:44.631152 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa260dfdb2e241e795368e69e1d3b1b5" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:44.631310 7142 raft_consensus.cc:2454] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:44.631381 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa260dfdb2e241e795368e69e1d3b1b5" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:44.631552 9415 raft_consensus.cc:2454] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:44.631655 9614 leader_election.cc:302] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:44.631919 14850 raft_consensus.cc:2781] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:44.631978 14850 raft_consensus.cc:482] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:44.632019 14850 raft_consensus.cc:3037] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.633153 14850 raft_consensus.cc:504] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.633533 14850 leader_election.cc:288] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:44.634307 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa260dfdb2e241e795368e69e1d3b1b5" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:44.634317 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa260dfdb2e241e795368e69e1d3b1b5" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:44.634429 9415 raft_consensus.cc:3037] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.634435 7142 raft_consensus.cc:3037] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.635321 9415 raft_consensus.cc:2454] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:44.635329 7142 raft_consensus.cc:2454] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:44.635700 9614 leader_election.cc:302] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:44.635977 14850 raft_consensus.cc:2781] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:44.636081 14850 raft_consensus.cc:686] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:44.636230 14850 consensus_queue.cc:237] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:44.637929 9672 consensus_queue.cc:237] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:44.639061 9415 raft_consensus.cc:1261] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:44.639178 9549 raft_consensus.cc:1261] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:44.639401 14850 consensus_queue.cc:1035] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:44.639588 14835 consensus_queue.cc:1035] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:44.639569 6731 catalog_manager.cc:5484] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:44.641228 14361 raft_consensus.cc:2932] T e856ff2c8ffb4bd4b44c694047c8a8fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:44.642151 9549 raft_consensus.cc:2932] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:44.642716 9415 raft_consensus.cc:2932] T e856ff2c8ffb4bd4b44c694047c8a8fa P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:44.645143 6731 catalog_manager.cc:5484] T e856ff2c8ffb4bd4b44c694047c8a8fa P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:44.650166 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet e856ff2c8ffb4bd4b44c694047c8a8fa with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:44.650534 14926 tablet_replica.cc:330] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:44.650735 14926 raft_consensus.cc:2229] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:44.651013 14926 raft_consensus.cc:2258] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:44.651597 14926 ts_tablet_manager.cc:1875] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:44.653985 14926 ts_tablet_manager.cc:1888] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:44.654142 14926 log.cc:1196] T e856ff2c8ffb4bd4b44c694047c8a8fa P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/e856ff2c8ffb4bd4b44c694047c8a8fa
I20240426 09:49:44.654955 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet e856ff2c8ffb4bd4b44c694047c8a8fa (table rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]) successfully deleted
I20240426 09:49:44.749076 6993 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:44.758682 6860 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:44.759943 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:44.762161 9266 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:44.765079 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:44.769881 9416 raft_consensus.cc:1261] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:44.770313 14850 consensus_queue.cc:1035] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.772172 14921 raft_consensus.cc:482] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:44.772280 14921 raft_consensus.cc:504] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.772514 7142 raft_consensus.cc:1261] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:44.772640 14921 leader_election.cc:288] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:44.773276 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360aa06557974c5ea635a32245c0903e" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:44.773766 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360aa06557974c5ea635a32245c0903e" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:44.773942 9549 raft_consensus.cc:2379] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1aaf8befba984e199e96f53f46b50441 in current term 1: Already voted for candidate 558c4ecd0ece460d9591aa7a3c799d19 in this term.
I20240426 09:49:44.774291 7077 leader_election.cc:302] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441; no voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:44.774379 6736 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet fa260dfdb2e241e795368e69e1d3b1b5 (table rebalancer_dynamic_table [id=d1c4d6597cfe43cc8cd4f0358381f9aa]). Expected version 1 got 0
I20240426 09:49:44.774528 14921 raft_consensus.cc:3037] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:44.775744 14921 raft_consensus.cc:2726] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240426 09:49:44.775907 14850 consensus_queue.cc:1035] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.777012 14977 tablet.cc:1722] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:44.777508 14977 tablet.cc:1620] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:44.779608 6736 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet fa260dfdb2e241e795368e69e1d3b1b5 (table rebalancer_dynamic_table [id=d1c4d6597cfe43cc8cd4f0358381f9aa]). Expected version 1 got 0
I20240426 09:49:44.783978 6739 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet fa260dfdb2e241e795368e69e1d3b1b5 (table rebalancer_dynamic_table [id=d1c4d6597cfe43cc8cd4f0358381f9aa]). Expected version 1 got 0
I20240426 09:49:44.787808 9395 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:44.788043 14975 mvcc.cc:205] Tried to move back new op lower bound from 7021055937612460032 to 7021055937071423488. Current Snapshot: MvccSnapshot[applied={T|T < 7021055937612460032}]
I20240426 09:49:44.788458 9660 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:44.793339 14980 tablet.cc:1722] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:44.793701 14980 tablet.cc:1620] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:44.799463 6736 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=d1c4d6597cfe43cc8cd4f0358381f9aa] alter complete (version 1)
I20240426 09:49:44.800011 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:44.811126 14835 consensus_queue.cc:1035] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.814559 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:44.821311 6736 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet fa260dfdb2e241e795368e69e1d3b1b5 (table rebalancer_dynamic_table [id=d1c4d6597cfe43cc8cd4f0358381f9aa]). Expected version 1 got 0
I20240426 09:49:44.829298 14319 consensus_queue.cc:1035] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.831696 14982 tablet.cc:1722] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:44.832525 14982 tablet.cc:1620] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:44.833173 14361 consensus_queue.cc:1035] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.842761 14319 consensus_queue.cc:1035] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.871160 9669 consensus_queue.cc:237] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:44.872480 9549 raft_consensus.cc:1261] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:44.872490 9416 raft_consensus.cc:1261] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:44.872879 14361 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:44.873046 14835 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:44.874943 14850 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:44.875365 9549 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:44.875384 9416 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:44.878652 6739 catalog_manager.cc:5484] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:44.881291 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet bbdcd6105a2647738257638e1e1096c3 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:44.881742 14853 tablet_replica.cc:330] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:44.881911 14853 raft_consensus.cc:2229] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:44.882055 14853 raft_consensus.cc:2258] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:44.882481 14853 ts_tablet_manager.cc:1875] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:44.884792 14853 ts_tablet_manager.cc:1888] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:44.884862 14853 log.cc:1196] T bbdcd6105a2647738257638e1e1096c3 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/bbdcd6105a2647738257638e1e1096c3
I20240426 09:49:44.887392 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet bbdcd6105a2647738257638e1e1096c3 (table rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]) successfully deleted
I20240426 09:49:44.908219 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:44.914978 14977 tablet.cc:1722] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:44.915333 14977 tablet.cc:1620] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:44.915773 14980 tablet.cc:1722] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:44.916090 14980 tablet.cc:1620] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:44.916376 14982 tablet.cc:1722] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:44.916641 14982 tablet.cc:1620] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:44.921872 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=d1c4d6597cfe43cc8cd4f0358381f9aa] alter complete (version 2)
I20240426 09:49:44.933720 14319 consensus_queue.cc:1035] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.945420 14885 consensus_queue.cc:1035] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:44.999751 6733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:45.001420 6993 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:45.009056 9265 tablet_service.cc:1467] Processing CreateTablet for tablet b5a4a67b5ab943b0a6299d74f2411190 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.009095 9660 tablet_service.cc:1467] Processing CreateTablet for tablet b5a4a67b5ab943b0a6299d74f2411190 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.009382 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a4a67b5ab943b0a6299d74f2411190. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.009382 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a4a67b5ab943b0a6299d74f2411190. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.011286 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 757210fd3040442e8195888c70666948 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.011608 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 757210fd3040442e8195888c70666948. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.011758 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 0c04f49b228c4c6d9bd6a222d8f287cc (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.011878 9395 tablet_service.cc:1467] Processing CreateTablet for tablet 0c04f49b228c4c6d9bd6a222d8f287cc (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.012032 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c04f49b228c4c6d9bd6a222d8f287cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.012120 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c04f49b228c4c6d9bd6a222d8f287cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.013780 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 0c04f49b228c4c6d9bd6a222d8f287cc (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.013852 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 757210fd3040442e8195888c70666948 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.014066 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 757210fd3040442e8195888c70666948. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.014073 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c04f49b228c4c6d9bd6a222d8f287cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.014449 6860 tablet_service.cc:1467] Processing CreateTablet for tablet b5a4a67b5ab943b0a6299d74f2411190 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.014740 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a4a67b5ab943b0a6299d74f2411190. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.015635 15062 tablet_bootstrap.cc:492] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:45.016883 15062 tablet_bootstrap.cc:654] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.017338 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 757210fd3040442e8195888c70666948 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.017603 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 757210fd3040442e8195888c70666948. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.018097 15063 tablet_bootstrap.cc:492] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:45.019706 15063 tablet_bootstrap.cc:654] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.020239 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:45.021468 15062 tablet_bootstrap.cc:492] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:45.021565 15062 ts_tablet_manager.cc:1367] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:45.022444 15062 raft_consensus.cc:348] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.022706 15062 raft_consensus.cc:374] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.022851 15062 raft_consensus.cc:729] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.022953 9395 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:45.023175 15062 consensus_queue.cc:260] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.023313 15066 tablet_bootstrap.cc:492] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:45.023520 15060 tablet_bootstrap.cc:492] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:45.023746 15064 tablet_bootstrap.cc:492] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:45.027396 15062 ts_tablet_manager.cc:1398] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:45.027889 15064 tablet_bootstrap.cc:654] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.028338 15060 tablet_bootstrap.cc:654] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.028647 15066 tablet_bootstrap.cc:654] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.029258 9266 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:45.031082 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:45.033176 15060 tablet_bootstrap.cc:492] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:45.033496 15066 tablet_bootstrap.cc:492] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:45.034296 15066 ts_tablet_manager.cc:1367] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:45.034448 15060 ts_tablet_manager.cc:1367] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:45.042714 14977 tablet.cc:1722] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:45.043705 9660 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:45.047292 14982 tablet.cc:1722] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:45.051162 15063 tablet_bootstrap.cc:492] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:45.051313 15063 ts_tablet_manager.cc:1367] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.002s
I20240426 09:49:45.051862 15063 raft_consensus.cc:348] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.051993 15063 raft_consensus.cc:374] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.052028 15063 raft_consensus.cc:729] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.052163 15063 consensus_queue.cc:260] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.052328 7255 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:45.052479 15063 ts_tablet_manager.cc:1398] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:45.052639 15063 tablet_bootstrap.cc:492] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:45.052635 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=d1c4d6597cfe43cc8cd4f0358381f9aa] alter complete (version 3)
I20240426 09:49:45.052881 14980 tablet.cc:1722] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:45.053691 15063 tablet_bootstrap.cc:654] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.055667 15061 tablet_bootstrap.cc:492] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:45.056489 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:45.057130 15061 tablet_bootstrap.cc:654] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.059540 15061 tablet_bootstrap.cc:492] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:45.059657 15061 ts_tablet_manager.cc:1367] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240426 09:49:45.060151 15061 raft_consensus.cc:348] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.060281 15061 raft_consensus.cc:374] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.060328 15061 raft_consensus.cc:729] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.060459 15061 consensus_queue.cc:260] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.060528 6860 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:45.060750 15061 ts_tablet_manager.cc:1398] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:45.060921 15061 tablet_bootstrap.cc:492] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:45.061923 15061 tablet_bootstrap.cc:654] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.064051 15066 raft_consensus.cc:348] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:45.063131 15060 raft_consensus.cc:348] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.064402 15066 raft_consensus.cc:374] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.064482 15060 raft_consensus.cc:374] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.064560 15066 raft_consensus.cc:729] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.064637 15060 raft_consensus.cc:729] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.064877 15060 consensus_queue.cc:260] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.065065 15066 consensus_queue.cc:260] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:45.075587 15061 tablet_bootstrap.cc:492] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:45.075752 15061 ts_tablet_manager.cc:1367] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240426 09:49:45.076342 15061 raft_consensus.cc:348] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:45.076463 15061 raft_consensus.cc:374] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.076490 15061 raft_consensus.cc:729] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.076608 15061 consensus_queue.cc:260] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:45.076910 15061 ts_tablet_manager.cc:1398] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:45.083998 15060 ts_tablet_manager.cc:1398] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.049s user 0.001s sys 0.000s
I20240426 09:49:45.084285 15066 ts_tablet_manager.cc:1398] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.023s user 0.001s sys 0.000s
I20240426 09:49:45.086761 15069 tablet_bootstrap.cc:492] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:45.087946 15069 tablet_bootstrap.cc:654] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.088137 15064 tablet_bootstrap.cc:492] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:45.088234 15064 ts_tablet_manager.cc:1367] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.000s
I20240426 09:49:45.088781 15064 raft_consensus.cc:348] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.088995 15064 raft_consensus.cc:374] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.089088 15064 raft_consensus.cc:729] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.089282 15064 consensus_queue.cc:260] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.089643 15064 ts_tablet_manager.cc:1398] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:45.090384 15069 tablet_bootstrap.cc:492] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:45.090476 15069 ts_tablet_manager.cc:1367] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:49:45.090997 15069 raft_consensus.cc:348] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.092017 15069 raft_consensus.cc:374] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.092088 15069 raft_consensus.cc:729] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.092242 15069 consensus_queue.cc:260] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.093315 15069 ts_tablet_manager.cc:1398] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:45.106088 15063 tablet_bootstrap.cc:492] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:45.106201 15063 ts_tablet_manager.cc:1367] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.054s user 0.000s sys 0.002s
I20240426 09:49:45.107893 15063 raft_consensus.cc:348] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:45.108022 15063 raft_consensus.cc:374] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.108052 15063 raft_consensus.cc:729] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.108165 15063 consensus_queue.cc:260] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:45.108426 15063 ts_tablet_manager.cc:1398] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:45.135958 7143 consensus_queue.cc:237] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.137221 9416 raft_consensus.cc:1261] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:45.137249 7011 raft_consensus.cc:1261] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:45.137590 13344 consensus_queue.cc:1035] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:45.137791 14921 consensus_queue.cc:1035] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:45.139536 13344 raft_consensus.cc:2932] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.139954 7011 raft_consensus.cc:2932] T f834d8a08b1341869fd644ff400fceae P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.142119 9416 raft_consensus.cc:2932] T f834d8a08b1341869fd644ff400fceae P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.143206 6733 catalog_manager.cc:5484] T f834d8a08b1341869fd644ff400fceae P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.143935 14796 raft_consensus.cc:482] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:45.144028 14796 raft_consensus.cc:504] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.144398 14796 leader_election.cc:288] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:45.144765 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c04f49b228c4c6d9bd6a222d8f287cc" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:45.144876 9416 raft_consensus.cc:2454] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:45.145154 7208 leader_election.cc:302] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:45.145153 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c04f49b228c4c6d9bd6a222d8f287cc" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:45.145303 9548 raft_consensus.cc:2454] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:45.145341 14796 raft_consensus.cc:2781] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:45.145388 14796 raft_consensus.cc:482] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:45.145434 14796 raft_consensus.cc:3037] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.146242 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet f834d8a08b1341869fd644ff400fceae with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:45.146442 14926 tablet_replica.cc:330] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:45.146582 14926 raft_consensus.cc:2229] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:45.146548 14796 raft_consensus.cc:504] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.146699 14926 raft_consensus.cc:2258] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.146925 14796 leader_election.cc:288] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:45.147378 14926 ts_tablet_manager.cc:1875] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:45.147611 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c04f49b228c4c6d9bd6a222d8f287cc" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:45.147704 9548 raft_consensus.cc:3037] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.148114 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c04f49b228c4c6d9bd6a222d8f287cc" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:45.148272 9416 raft_consensus.cc:3037] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.148659 9548 raft_consensus.cc:2454] T 0c04f49b228c4c6d9bd6a222d8f287cc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:45.149564 14926 ts_tablet_manager.cc:1888] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:45.149909 14926 log.cc:1196] T f834d8a08b1341869fd644ff400fceae P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/f834d8a08b1341869fd644ff400fceae
I20240426 09:49:45.151249 9416 raft_consensus.cc:2454] T 0c04f49b228c4c6d9bd6a222d8f287cc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:45.155838 6736 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:45.155889 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet f834d8a08b1341869fd644ff400fceae (table rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]) successfully deleted
I20240426 09:49:45.155982 6736 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:45.156635 7208 leader_election.cc:302] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:45.156960 14796 raft_consensus.cc:2781] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:45.157049 14796 raft_consensus.cc:686] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:45.157184 14796 consensus_queue.cc:237] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.159206 6736 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet fa260dfdb2e241e795368e69e1d3b1b5
I20240426 09:49:45.159449 6737 catalog_manager.cc:5484] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:45.160133 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet fa260dfdb2e241e795368e69e1d3b1b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:45 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:45.160457 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet fa260dfdb2e241e795368e69e1d3b1b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:45 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:45.160553 9395 tablet_service.cc:1514] Processing DeleteTablet for tablet fa260dfdb2e241e795368e69e1d3b1b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:45 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:45.161202 15107 tablet_replica.cc:330] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:45.161577 15107 raft_consensus.cc:2229] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:45.161798 15107 raft_consensus.cc:2258] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.162259 15108 tablet_replica.cc:330] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:45.162410 15106 tablet_replica.cc:330] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:45.162585 15106 raft_consensus.cc:2229] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:45.162631 15107 ts_tablet_manager.cc:1875] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:45.162801 15108 raft_consensus.cc:2229] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:45.162993 15106 raft_consensus.cc:2258] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.163177 15108 raft_consensus.cc:2258] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.165110 15107 ts_tablet_manager.cc:1888] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:45.165247 15107 log.cc:1196] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/fa260dfdb2e241e795368e69e1d3b1b5
I20240426 09:49:45.165620 15107 ts_tablet_manager.cc:1909] T fa260dfdb2e241e795368e69e1d3b1b5 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:45.165905 15108 ts_tablet_manager.cc:1875] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:45.166062 15106 ts_tablet_manager.cc:1875] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:45.166384 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet fa260dfdb2e241e795368e69e1d3b1b5 (table rebalancer_dynamic_table_new_name [id=d1c4d6597cfe43cc8cd4f0358381f9aa]) successfully deleted
I20240426 09:49:45.168280 15108 ts_tablet_manager.cc:1888] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:45.168352 15108 log.cc:1196] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/fa260dfdb2e241e795368e69e1d3b1b5
I20240426 09:49:45.168587 15108 ts_tablet_manager.cc:1909] T fa260dfdb2e241e795368e69e1d3b1b5 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:45.169256 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet fa260dfdb2e241e795368e69e1d3b1b5 (table rebalancer_dynamic_table_new_name [id=d1c4d6597cfe43cc8cd4f0358381f9aa]) successfully deleted
I20240426 09:49:45.169605 14441 raft_consensus.cc:482] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:45.169694 14441 raft_consensus.cc:504] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:45.170105 14441 leader_election.cc:288] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:45.170600 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757210fd3040442e8195888c70666948" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:45.170719 9286 raft_consensus.cc:2454] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:45.171336 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757210fd3040442e8195888c70666948" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:45.171489 7275 raft_consensus.cc:2454] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:45.171494 15106 ts_tablet_manager.cc:1888] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:45.171562 15106 log.cc:1196] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/fa260dfdb2e241e795368e69e1d3b1b5
I20240426 09:49:45.171891 15106 ts_tablet_manager.cc:1909] T fa260dfdb2e241e795368e69e1d3b1b5 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:45.172237 6947 leader_election.cc:302] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:45.172720 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet fa260dfdb2e241e795368e69e1d3b1b5 (table rebalancer_dynamic_table_new_name [id=d1c4d6597cfe43cc8cd4f0358381f9aa]) successfully deleted
I20240426 09:49:45.173069 14441 raft_consensus.cc:2781] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:45.173144 14441 raft_consensus.cc:482] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:45.173188 14441 raft_consensus.cc:3037] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.174166 14441 raft_consensus.cc:504] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:45.174479 14441 leader_election.cc:288] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:45.175020 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757210fd3040442e8195888c70666948" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:45.175143 9286 raft_consensus.cc:3037] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.175123 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "757210fd3040442e8195888c70666948" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:45.175233 7275 raft_consensus.cc:3037] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.176081 9286 raft_consensus.cc:2454] T 757210fd3040442e8195888c70666948 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:45.176081 7275 raft_consensus.cc:2454] T 757210fd3040442e8195888c70666948 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:45.176383 6948 leader_election.cc:302] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:45.176590 14441 raft_consensus.cc:2781] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:45.176663 14441 raft_consensus.cc:686] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:45.176782 14441 consensus_queue.cc:237] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:45.179083 6737 catalog_manager.cc:5484] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:45.195812 14319 raft_consensus.cc:482] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:45.195964 14319 raft_consensus.cc:504] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.196465 14319 leader_election.cc:288] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:45.196758 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a4a67b5ab943b0a6299d74f2411190" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:45.196906 9669 raft_consensus.cc:2454] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:45.196928 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a4a67b5ab943b0a6299d74f2411190" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:45.197057 9286 raft_consensus.cc:2454] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:45.197244 6815 leader_election.cc:302] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:45.197410 14319 raft_consensus.cc:2781] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:45.197474 14319 raft_consensus.cc:482] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:45.197525 14319 raft_consensus.cc:3037] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.198658 14319 raft_consensus.cc:504] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.199044 14319 leader_election.cc:288] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:45.199411 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a4a67b5ab943b0a6299d74f2411190" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:45.199424 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5a4a67b5ab943b0a6299d74f2411190" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:45.199534 9286 raft_consensus.cc:3037] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.199553 9669 raft_consensus.cc:3037] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.200474 9286 raft_consensus.cc:2454] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:45.200584 9669 raft_consensus.cc:2454] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:45.200757 6816 leader_election.cc:302] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:45.200918 14319 raft_consensus.cc:2781] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:45.200989 14319 raft_consensus.cc:686] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:45.201128 14319 consensus_queue.cc:237] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.203897 6737 catalog_manager.cc:5484] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:45.264993 6735 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:45.266155 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:45.271831 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:45.272078 6993 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:45.282236 9395 tablet_service.cc:1467] Processing CreateTablet for tablet 841889520242480c8ce39a859ffaf476 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3324e64189354bcab90f9ff59e38fce1]), partition=
I20240426 09:49:45.282539 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 841889520242480c8ce39a859ffaf476 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3324e64189354bcab90f9ff59e38fce1]), partition=
I20240426 09:49:45.282567 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 841889520242480c8ce39a859ffaf476. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.282660 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 841889520242480c8ce39a859ffaf476 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3324e64189354bcab90f9ff59e38fce1]), partition=
I20240426 09:49:45.282815 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 841889520242480c8ce39a859ffaf476. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.282949 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 841889520242480c8ce39a859ffaf476. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.283089 9660 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:45.284777 15062 tablet_bootstrap.cc:492] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:45.285010 15063 tablet_bootstrap.cc:492] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:45.285609 15062 tablet_bootstrap.cc:654] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.286007 15063 tablet_bootstrap.cc:654] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.286639 15061 tablet_bootstrap.cc:492] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:45.286675 15062 tablet_bootstrap.cc:492] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:45.286756 15062 ts_tablet_manager.cc:1367] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:45.287175 15062 raft_consensus.cc:348] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.287261 15062 raft_consensus.cc:374] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.287281 15062 raft_consensus.cc:729] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.287360 15062 consensus_queue.cc:260] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.287508 15061 tablet_bootstrap.cc:654] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.287642 15062 ts_tablet_manager.cc:1398] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:45.289723 15061 tablet_bootstrap.cc:492] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:45.289801 15061 ts_tablet_manager.cc:1367] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:45.290195 15061 raft_consensus.cc:348] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.290295 15061 raft_consensus.cc:374] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.290333 15061 raft_consensus.cc:729] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.290445 15061 consensus_queue.cc:260] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.290688 15061 ts_tablet_manager.cc:1398] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:45.291829 15063 tablet_bootstrap.cc:492] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:45.291920 15063 ts_tablet_manager.cc:1367] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240426 09:49:45.292410 15063 raft_consensus.cc:348] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.292532 15063 raft_consensus.cc:374] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.292572 15063 raft_consensus.cc:729] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.292686 15063 consensus_queue.cc:260] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.293792 15063 ts_tablet_manager.cc:1398] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:45.294167 9395 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:45.294482 7255 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:45.307296 9266 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:45.314728 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:45.334417 14799 raft_consensus.cc:482] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:45.334543 14799 raft_consensus.cc:504] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.334968 14799 leader_election.cc:288] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:45.335492 13569 raft_consensus.cc:482] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:45.335623 13569 raft_consensus.cc:504] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.336048 13569 leader_election.cc:288] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:45.336211 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "841889520242480c8ce39a859ffaf476" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:45.336318 9416 raft_consensus.cc:2454] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:45.336519 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "841889520242480c8ce39a859ffaf476" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:45.336630 7275 raft_consensus.cc:2454] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:45.337339 9221 leader_election.cc:302] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:45.338137 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "841889520242480c8ce39a859ffaf476" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:45.338251 9286 raft_consensus.cc:2454] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:45.338524 9351 leader_election.cc:302] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:45.340029 14799 raft_consensus.cc:2781] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:45.340096 14799 raft_consensus.cc:482] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:45.340135 14799 raft_consensus.cc:3037] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.341297 14799 raft_consensus.cc:504] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.341634 14799 leader_election.cc:288] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:45.342741 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "841889520242480c8ce39a859ffaf476" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:45.342914 7275 raft_consensus.cc:3037] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.343433 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "841889520242480c8ce39a859ffaf476" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:45.343539 9416 raft_consensus.cc:3037] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.344063 7275 raft_consensus.cc:2454] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:45.344565 9416 raft_consensus.cc:2454] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:45.345013 13569 raft_consensus.cc:2741] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20240426 09:49:45.345048 9221 leader_election.cc:302] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:45.346339 14799 raft_consensus.cc:2781] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:45.346515 14799 raft_consensus.cc:686] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:45.346650 14799 consensus_queue.cc:237] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.349020 6735 catalog_manager.cc:5484] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.359694 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "841889520242480c8ce39a859ffaf476" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:45.359879 7275 raft_consensus.cc:2379] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3b41d87e7bfb419f8aa7500100129b6c in current term 1: Already voted for candidate a692602e1a2e4794877e6fb92a7b9962 in this term.
I20240426 09:49:45.394167 6882 consensus_queue.cc:237] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.395582 9416 raft_consensus.cc:1261] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:45.395677 7011 raft_consensus.cc:1261] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:45.396082 14319 consensus_queue.cc:1035] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:45.396281 14885 consensus_queue.cc:1035] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:45.398123 15178 raft_consensus.cc:2932] T a42302cab4524e7a85defe4da40be268 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.398470 7011 raft_consensus.cc:2932] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.398566 9416 raft_consensus.cc:2932] T a42302cab4524e7a85defe4da40be268 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.401441 6735 catalog_manager.cc:5484] T a42302cab4524e7a85defe4da40be268 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.403923 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet a42302cab4524e7a85defe4da40be268 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:45.404157 14926 tablet_replica.cc:330] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:45.404315 14926 raft_consensus.cc:2229] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:45.404455 14926 raft_consensus.cc:2258] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.404922 14926 ts_tablet_manager.cc:1875] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:45.406510 14926 ts_tablet_manager.cc:1888] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:45.406585 14926 log.cc:1196] T a42302cab4524e7a85defe4da40be268 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/a42302cab4524e7a85defe4da40be268
I20240426 09:49:45.407079 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet a42302cab4524e7a85defe4da40be268 (table rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]) successfully deleted
I20240426 09:49:45.463229 6733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:45.469866 7275 raft_consensus.cc:1261] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:45.470530 9416 raft_consensus.cc:1261] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:45.471515 15158 consensus_queue.cc:1035] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:45.475145 15158 consensus_queue.cc:1035] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:45.476801 6733 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 841889520242480c8ce39a859ffaf476 (table rebalancer_dynamic_table [id=3324e64189354bcab90f9ff59e38fce1]). Expected version 1 got 0
I20240426 09:49:45.477535 14980 tablet.cc:1722] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:45.477857 14980 tablet.cc:1620] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:45.478595 6733 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 841889520242480c8ce39a859ffaf476 (table rebalancer_dynamic_table [id=3324e64189354bcab90f9ff59e38fce1]). Expected version 1 got 0
I20240426 09:49:45.479224 15224 tablet.cc:1722] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:45.481341 15224 tablet.cc:1620] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:45.483845 6736 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=3324e64189354bcab90f9ff59e38fce1] alter complete (version 1)
I20240426 09:49:45.486929 6733 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 841889520242480c8ce39a859ffaf476 (table rebalancer_dynamic_table [id=3324e64189354bcab90f9ff59e38fce1]). Expected version 1 got 0
I20240426 09:49:45.487720 15222 mvcc.cc:205] Tried to move back new op lower bound from 7021055940479270912 to 7021055939981365248. Current Snapshot: MvccSnapshot[applied={T|T < 7021055940479270912}]
I20240426 09:49:45.493229 15225 tablet.cc:1722] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:45.493587 15225 tablet.cc:1620] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:45.517189 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:45.522275 9266 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:45.522861 6993 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:45.522890 9660 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:45.526320 9395 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:45.526594 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:45.529803 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:45.549495 7255 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:45.590967 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:45.598114 15224 tablet.cc:1722] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:45.598506 15224 tablet.cc:1620] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:45.598836 15225 tablet.cc:1722] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:45.599131 15225 tablet.cc:1620] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:45.605708 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=3324e64189354bcab90f9ff59e38fce1] alter complete (version 2)
I20240426 09:49:45.606061 14980 tablet.cc:1722] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:45.606346 14980 tablet.cc:1620] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:45.607585 14319 consensus_queue.cc:1035] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:45.612179 14885 consensus_queue.cc:1035] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:45.613760 15110 consensus_queue.cc:1035] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:45.616503 14441 consensus_queue.cc:1035] T 757210fd3040442e8195888c70666948 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:45.637650 7275 consensus_queue.cc:237] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:45.638932 9672 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:45.638931 6882 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:45.639333 14796 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:45.639525 15261 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:45.641247 15261 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:45.641692 6882 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:45.642719 9672 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:45.644917 6737 catalog_manager.cc:5484] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:45.647539 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 4189112964fb425aac074352dd873415 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:45.647913 15264 tablet_replica.cc:330] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:45.648088 15264 raft_consensus.cc:2229] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:45.648249 15264 raft_consensus.cc:2258] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.649123 15264 ts_tablet_manager.cc:1875] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:45.651288 15264 ts_tablet_manager.cc:1888] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:45.651435 15264 log.cc:1196] T 4189112964fb425aac074352dd873415 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/4189112964fb425aac074352dd873415
I20240426 09:49:45.652050 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 4189112964fb425aac074352dd873415 (table rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]) successfully deleted
I20240426 09:49:45.699539 15261 consensus_queue.cc:1035] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:45.704566 14398 consensus_queue.cc:1035] T 0c04f49b228c4c6d9bd6a222d8f287cc P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:45.709327 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:45.718242 15224 tablet.cc:1722] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:45.723635 14980 tablet.cc:1722] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:45.724195 15225 tablet.cc:1722] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:45.731238 6736 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=3324e64189354bcab90f9ff59e38fce1] alter complete (version 3)
I20240426 09:49:45.739272 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:45.750507 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 6dce6f1e165c492ab23022527bac580a (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.750558 6993 tablet_service.cc:1467] Processing CreateTablet for tablet c6956266cb444536945540ecdbaba28e (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.750852 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6956266cb444536945540ecdbaba28e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.750852 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dce6f1e165c492ab23022527bac580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.750932 9395 tablet_service.cc:1467] Processing CreateTablet for tablet 6dce6f1e165c492ab23022527bac580a (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.750995 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 6dce6f1e165c492ab23022527bac580a (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.751236 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dce6f1e165c492ab23022527bac580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.751263 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dce6f1e165c492ab23022527bac580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.753542 7253 tablet_service.cc:1467] Processing CreateTablet for tablet c6956266cb444536945540ecdbaba28e (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.753780 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6956266cb444536945540ecdbaba28e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.754042 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 5040b4c862a84a5786857fb0f53849ca (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.754320 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5040b4c862a84a5786857fb0f53849ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.755861 9397 tablet_service.cc:1467] Processing CreateTablet for tablet c6956266cb444536945540ecdbaba28e (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.756088 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6956266cb444536945540ecdbaba28e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.756896 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 5040b4c862a84a5786857fb0f53849ca (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.756979 15312 tablet_bootstrap.cc:492] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:45.757153 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5040b4c862a84a5786857fb0f53849ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.758296 15062 tablet_bootstrap.cc:492] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:45.759289 15062 tablet_bootstrap.cc:654] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.759922 15063 tablet_bootstrap.cc:492] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:45.760120 15312 tablet_bootstrap.cc:654] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.760875 15063 tablet_bootstrap.cc:654] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.762240 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 5040b4c862a84a5786857fb0f53849ca (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:45.762485 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5040b4c862a84a5786857fb0f53849ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.764807 15316 tablet_bootstrap.cc:492] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:45.767848 15316 tablet_bootstrap.cc:654] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.770131 15063 tablet_bootstrap.cc:492] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:45.770233 15063 ts_tablet_manager.cc:1367] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240426 09:49:45.770298 15316 tablet_bootstrap.cc:492] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:45.770431 15316 ts_tablet_manager.cc:1367] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:45.770730 15063 raft_consensus.cc:348] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.770882 15063 raft_consensus.cc:374] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.770922 15063 raft_consensus.cc:729] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.771051 15063 consensus_queue.cc:260] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.771378 15063 ts_tablet_manager.cc:1398] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:45.771561 15063 tablet_bootstrap.cc:492] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:45.772064 15316 raft_consensus.cc:348] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.772271 15316 raft_consensus.cc:374] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.772428 15316 raft_consensus.cc:729] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.772495 15063 tablet_bootstrap.cc:654] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.772751 15316 consensus_queue.cc:260] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.773391 15316 ts_tablet_manager.cc:1398] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:45.774291 15062 tablet_bootstrap.cc:492] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:45.774389 15062 ts_tablet_manager.cc:1367] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240426 09:49:45.774940 15062 raft_consensus.cc:348] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.775095 15062 raft_consensus.cc:374] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.775137 15062 raft_consensus.cc:729] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.775264 15062 consensus_queue.cc:260] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.775549 15062 ts_tablet_manager.cc:1398] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:45.775805 15062 tablet_bootstrap.cc:492] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:45.776413 15310 tablet_bootstrap.cc:492] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:45.776876 15062 tablet_bootstrap.cc:654] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.779072 15063 tablet_bootstrap.cc:492] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:45.779273 15063 ts_tablet_manager.cc:1367] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:45.779770 15063 raft_consensus.cc:348] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.780328 15311 tablet_bootstrap.cc:492] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:45.780508 15310 tablet_bootstrap.cc:654] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.782333 15311 tablet_bootstrap.cc:654] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.782855 15063 raft_consensus.cc:374] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.782918 15063 raft_consensus.cc:729] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.783047 15063 consensus_queue.cc:260] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.783942 15063 ts_tablet_manager.cc:1398] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:49:45.786022 15062 tablet_bootstrap.cc:492] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:45.786168 15062 ts_tablet_manager.cc:1367] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20240426 09:49:45.790706 15310 tablet_bootstrap.cc:492] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:45.790895 15310 ts_tablet_manager.cc:1367] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240426 09:49:45.791476 15310 raft_consensus.cc:348] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.791656 15310 raft_consensus.cc:374] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.791728 15310 raft_consensus.cc:729] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.791918 15310 consensus_queue.cc:260] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.792290 15310 ts_tablet_manager.cc:1398] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:45.792521 15310 tablet_bootstrap.cc:492] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:45.793730 6993 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:45.794205 15310 tablet_bootstrap.cc:654] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.796896 15062 raft_consensus.cc:348] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.800513 15062 raft_consensus.cc:374] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.800627 15062 raft_consensus.cc:729] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.800828 15062 consensus_queue.cc:260] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.801209 15062 ts_tablet_manager.cc:1398] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.013s user 0.000s sys 0.001s
I20240426 09:49:45.802793 15310 tablet_bootstrap.cc:492] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:45.803447 15310 ts_tablet_manager.cc:1367] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.001s
I20240426 09:49:45.804160 15310 raft_consensus.cc:348] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.804345 15312 tablet_bootstrap.cc:492] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:45.804459 15312 ts_tablet_manager.cc:1367] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.048s user 0.000s sys 0.003s
I20240426 09:49:45.804608 15310 raft_consensus.cc:374] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.804682 15310 raft_consensus.cc:729] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.804813 15310 consensus_queue.cc:260] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.804924 15312 raft_consensus.cc:348] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.805052 15312 raft_consensus.cc:374] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.805084 15312 raft_consensus.cc:729] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.805186 15312 consensus_queue.cc:260] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:45.805426 15312 ts_tablet_manager.cc:1398] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:45.807672 15310 ts_tablet_manager.cc:1398] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240426 09:49:45.808218 9527 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:45.809715 15311 tablet_bootstrap.cc:492] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:45.809865 15311 ts_tablet_manager.cc:1367] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20240426 09:49:45.810310 9266 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:45.810580 15311 raft_consensus.cc:348] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.810775 15311 raft_consensus.cc:374] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.810854 15311 raft_consensus.cc:729] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.811013 15311 consensus_queue.cc:260] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.811959 15311 ts_tablet_manager.cc:1398] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:45.812675 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:45.813882 9660 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:45.826994 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:45.834508 7253 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:45.835644 13569 raft_consensus.cc:482] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:45.835742 13569 raft_consensus.cc:504] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.835840 9397 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240426 09:49:45.836113 13569 leader_election.cc:288] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:45.836799 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dce6f1e165c492ab23022527bac580a" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:45.836920 9672 raft_consensus.cc:2454] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:45.837118 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dce6f1e165c492ab23022527bac580a" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:45.837203 9350 leader_election.cc:302] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:45.837240 7274 raft_consensus.cc:2454] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:45.837355 13569 raft_consensus.cc:2781] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:45.837393 13569 raft_consensus.cc:482] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:45.837426 13569 raft_consensus.cc:3037] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.838450 13569 raft_consensus.cc:504] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.838817 13569 leader_election.cc:288] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:45.839787 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dce6f1e165c492ab23022527bac580a" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:45.839897 9672 raft_consensus.cc:3037] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.840831 9672 raft_consensus.cc:2454] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:45.841112 9350 leader_election.cc:302] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:45.841720 13569 raft_consensus.cc:2781] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:45.841759 6736 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:45.841874 13569 raft_consensus.cc:686] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:45.841872 6736 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:45.842020 13569 consensus_queue.cc:237] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:45.843890 14796 raft_consensus.cc:482] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:45.843993 14796 raft_consensus.cc:504] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.844091 6736 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 841889520242480c8ce39a859ffaf476
I20240426 09:49:45.844354 14796 leader_election.cc:288] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:45.845379 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6956266cb444536945540ecdbaba28e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:45.845496 7013 raft_consensus.cc:2454] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:45.845513 6732 catalog_manager.cc:5484] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:45.845618 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 841889520242480c8ce39a859ffaf476 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:45 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:45.845932 7208 leader_election.cc:302] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:45.846031 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet 841889520242480c8ce39a859ffaf476 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:45 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:45.846099 14796 raft_consensus.cc:2781] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:45.846139 14796 raft_consensus.cc:482] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:45.846171 14796 raft_consensus.cc:3037] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.846190 14926 tablet_replica.cc:330] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:45.846336 14926 raft_consensus.cc:2229] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:45.846457 14926 raft_consensus.cc:2258] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.847055 14926 ts_tablet_manager.cc:1875] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:45.847189 14796 raft_consensus.cc:504] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.847553 14796 leader_election.cc:288] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:45.848040 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6956266cb444536945540ecdbaba28e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:45.848147 9417 raft_consensus.cc:3037] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.849043 9417 raft_consensus.cc:2454] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:45.849107 14926 ts_tablet_manager.cc:1888] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:45.849181 14926 log.cc:1196] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/841889520242480c8ce39a859ffaf476
I20240426 09:49:45.849570 14926 ts_tablet_manager.cc:1909] T 841889520242480c8ce39a859ffaf476 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:45.850240 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 841889520242480c8ce39a859ffaf476 (table rebalancer_dynamic_table_new_name [id=3324e64189354bcab90f9ff59e38fce1]) successfully deleted
I20240426 09:49:45.850270 7208 leader_election.cc:302] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:45.850593 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6956266cb444536945540ecdbaba28e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:45.850704 7013 raft_consensus.cc:3037] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.850822 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 841889520242480c8ce39a859ffaf476 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:45 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:45.851656 7013 raft_consensus.cc:2454] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:45.853469 15337 tablet_replica.cc:330] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:45.853627 15337 raft_consensus.cc:2229] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:45.853838 15337 raft_consensus.cc:2258] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.860487 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dce6f1e165c492ab23022527bac580a" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:45.860626 7274 raft_consensus.cc:3037] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:45.861683 7274 raft_consensus.cc:2454] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:45.862411 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6956266cb444536945540ecdbaba28e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:45.862525 9416 raft_consensus.cc:2362] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1. Re-sending same reply.
I20240426 09:49:45.863055 14796 raft_consensus.cc:2781] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:45.863138 14796 raft_consensus.cc:686] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:45.863260 14796 consensus_queue.cc:237] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.865407 6732 catalog_manager.cc:5484] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:45.875355 15337 ts_tablet_manager.cc:1875] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:45.878086 15337 ts_tablet_manager.cc:1888] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:45.878175 15337 log.cc:1196] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/841889520242480c8ce39a859ffaf476
I20240426 09:49:45.878443 15337 ts_tablet_manager.cc:1909] T 841889520242480c8ce39a859ffaf476 P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:45.879191 15336 tablet_replica.cc:330] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:45.879189 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 841889520242480c8ce39a859ffaf476 (table rebalancer_dynamic_table_new_name [id=3324e64189354bcab90f9ff59e38fce1]) successfully deleted
I20240426 09:49:45.879432 15336 raft_consensus.cc:2229] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:45.879597 15336 raft_consensus.cc:2258] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.880147 15336 ts_tablet_manager.cc:1875] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:45.882283 15336 ts_tablet_manager.cc:1888] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:45.882362 15336 log.cc:1196] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/841889520242480c8ce39a859ffaf476
I20240426 09:49:45.882696 15336 ts_tablet_manager.cc:1909] T 841889520242480c8ce39a859ffaf476 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:45.883427 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 841889520242480c8ce39a859ffaf476 (table rebalancer_dynamic_table_new_name [id=3324e64189354bcab90f9ff59e38fce1]) successfully deleted
I20240426 09:49:45.907151 7013 consensus_queue.cc:237] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:45.908378 9672 raft_consensus.cc:1261] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:49:45.908479 7143 raft_consensus.cc:1261] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:49:45.908771 15110 consensus_queue.cc:1035] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:45.908954 14441 consensus_queue.cc:1035] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:45.910673 15237 raft_consensus.cc:2932] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:45.911068 7143 raft_consensus.cc:2932] T cd67a40e0b834a83b41259ee4da3f3db P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:45.912923 9672 raft_consensus.cc:2932] T cd67a40e0b834a83b41259ee4da3f3db P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:45.914497 6732 catalog_manager.cc:5484] T cd67a40e0b834a83b41259ee4da3f3db P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 2 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:45.917519 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet cd67a40e0b834a83b41259ee4da3f3db with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:45.918190 15357 tablet_replica.cc:330] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:45.918398 15357 raft_consensus.cc:2229] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:45.918601 15357 raft_consensus.cc:2258] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:45.919085 15357 ts_tablet_manager.cc:1875] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:45.921030 15357 ts_tablet_manager.cc:1888] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240426 09:49:45.921093 15357 log.cc:1196] T cd67a40e0b834a83b41259ee4da3f3db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/cd67a40e0b834a83b41259ee4da3f3db
I20240426 09:49:45.921859 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet cd67a40e0b834a83b41259ee4da3f3db (table rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]) successfully deleted
I20240426 09:49:45.951756 6736 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:45.957264 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 06f9ef6c123f494bb8f72a034267fe29 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9fd7fb73b74f43e1b15ded14256f18e4]), partition=
I20240426 09:49:45.957554 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f9ef6c123f494bb8f72a034267fe29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.959074 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 06f9ef6c123f494bb8f72a034267fe29 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9fd7fb73b74f43e1b15ded14256f18e4]), partition=
I20240426 09:49:45.959308 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f9ef6c123f494bb8f72a034267fe29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.959781 15378 tablet_bootstrap.cc:492] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:45.960662 15378 tablet_bootstrap.cc:654] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.961295 15316 tablet_bootstrap.cc:492] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:45.962141 15316 tablet_bootstrap.cc:654] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.963228 15316 tablet_bootstrap.cc:492] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:45.963344 15316 ts_tablet_manager.cc:1367] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:45.963829 15316 raft_consensus.cc:348] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.963943 15316 raft_consensus.cc:374] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.963984 15316 raft_consensus.cc:729] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.964069 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 06f9ef6c123f494bb8f72a034267fe29 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9fd7fb73b74f43e1b15ded14256f18e4]), partition=
I20240426 09:49:45.964100 15316 consensus_queue.cc:260] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.964280 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06f9ef6c123f494bb8f72a034267fe29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:45.964449 15316 ts_tablet_manager.cc:1398] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:45.965878 15378 tablet_bootstrap.cc:492] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:45.966023 15378 ts_tablet_manager.cc:1367] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:45.967957 15378 raft_consensus.cc:348] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.968140 15378 raft_consensus.cc:374] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.968219 15378 raft_consensus.cc:729] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.968469 15378 consensus_queue.cc:260] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.969142 15378 ts_tablet_manager.cc:1398] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:45.970963 15386 tablet_bootstrap.cc:492] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:45.972834 15386 tablet_bootstrap.cc:654] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:45.975425 15386 tablet_bootstrap.cc:492] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:45.975508 15386 ts_tablet_manager.cc:1367] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:45.975685 7274 consensus_queue.cc:237] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:45.975912 15386 raft_consensus.cc:348] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.976009 15386 raft_consensus.cc:374] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:45.976053 15386 raft_consensus.cc:729] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:45.976188 15386 consensus_queue.cc:260] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:45.976531 15386 ts_tablet_manager.cc:1398] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:45.977221 7013 raft_consensus.cc:1261] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:45.977280 7143 raft_consensus.cc:1261] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:45.977628 15261 consensus_queue.cc:1035] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:49:45.978183 7208 consensus_peers.cc:485] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 325e1d125617456f877204314d9e7bb9. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:45.978279 15261 consensus_queue.cc:1035] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:45.980870 14796 raft_consensus.cc:2932] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:45.981280 7013 raft_consensus.cc:2932] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:45.982915 7143 raft_consensus.cc:2932] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:45.985694 6736 catalog_manager.cc:5484] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:46.003901 14802 raft_consensus.cc:482] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:46.004179 14802 raft_consensus.cc:504] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:46.005138 14802 leader_election.cc:288] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:46.006388 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5040b4c862a84a5786857fb0f53849ca" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:46.006486 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5040b4c862a84a5786857fb0f53849ca" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:46.006565 9672 raft_consensus.cc:2454] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:46.006628 7142 raft_consensus.cc:2454] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:46.006935 9482 leader_election.cc:302] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:46.007112 14802 raft_consensus.cc:2781] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:46.007171 14802 raft_consensus.cc:482] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:46.007205 14802 raft_consensus.cc:3037] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.008271 14802 raft_consensus.cc:504] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:46.008608 14802 leader_election.cc:288] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:46.008929 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5040b4c862a84a5786857fb0f53849ca" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:46.008946 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5040b4c862a84a5786857fb0f53849ca" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:46.009025 7142 raft_consensus.cc:3037] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.009050 9672 raft_consensus.cc:3037] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.009848 9672 raft_consensus.cc:2454] T 5040b4c862a84a5786857fb0f53849ca P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:46.009954 7142 raft_consensus.cc:2454] T 5040b4c862a84a5786857fb0f53849ca P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:46.010104 9482 leader_election.cc:302] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:46.010371 14802 raft_consensus.cc:2781] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:46.010468 14802 raft_consensus.cc:686] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:46.010627 14802 consensus_queue.cc:237] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:46.012835 6736 catalog_manager.cc:5484] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:46.014463 15309 raft_consensus.cc:482] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:46.014559 15309 raft_consensus.cc:504] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.014921 15309 leader_election.cc:288] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:46.015339 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f9ef6c123f494bb8f72a034267fe29" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:46.015478 6882 raft_consensus.cc:2454] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:46.015548 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f9ef6c123f494bb8f72a034267fe29" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:46.015664 9547 raft_consensus.cc:2454] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:46.015750 9221 leader_election.cc:302] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:46.015904 15309 raft_consensus.cc:2781] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:46.015951 15309 raft_consensus.cc:482] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:46.016000 15309 raft_consensus.cc:3037] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.016906 15309 raft_consensus.cc:504] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.017222 15309 leader_election.cc:288] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:46.017503 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f9ef6c123f494bb8f72a034267fe29" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:46.017553 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06f9ef6c123f494bb8f72a034267fe29" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:46.017623 6882 raft_consensus.cc:3037] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.017673 9547 raft_consensus.cc:3037] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.018507 6882 raft_consensus.cc:2454] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:46.018684 9547 raft_consensus.cc:2454] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:46.018797 9221 leader_election.cc:302] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:46.019034 15309 raft_consensus.cc:2781] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:46.019114 15309 raft_consensus.cc:686] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:46.019237 15309 consensus_queue.cc:237] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.021101 6736 catalog_manager.cc:5484] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:46.090862 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:46.099969 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:46.100354 9397 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:46.107898 9266 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:46.109632 9527 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:46.114262 6993 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:46.116035 9660 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:46.121817 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:46.130967 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:46.137290 6882 raft_consensus.cc:1261] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:46.139133 15309 consensus_queue.cc:1035] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:46.141706 9547 raft_consensus.cc:1261] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:46.143399 15309 consensus_queue.cc:1035] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:46.146935 6732 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 06f9ef6c123f494bb8f72a034267fe29 (table rebalancer_dynamic_table [id=9fd7fb73b74f43e1b15ded14256f18e4]). Expected version 1 got 0
I20240426 09:49:46.147984 6740 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet 06f9ef6c123f494bb8f72a034267fe29 (table rebalancer_dynamic_table [id=9fd7fb73b74f43e1b15ded14256f18e4]). Expected version 1 got 0
I20240426 09:49:46.148134 15224 tablet.cc:1722] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:46.148444 15224 tablet.cc:1620] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:46.151279 15451 tablet.cc:1722] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:46.151629 15451 tablet.cc:1620] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:46.161001 6740 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=9fd7fb73b74f43e1b15ded14256f18e4] alter complete (version 1)
I20240426 09:49:46.177919 6732 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 06f9ef6c123f494bb8f72a034267fe29 (table rebalancer_dynamic_table [id=9fd7fb73b74f43e1b15ded14256f18e4]). Expected version 1 got 0
I20240426 09:49:46.182921 15452 tablet.cc:1722] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:46.183261 15452 tablet.cc:1620] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:46.264037 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:46.274116 15224 tablet.cc:1722] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:46.274484 15224 tablet.cc:1620] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:46.276036 15451 tablet.cc:1722] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:46.276326 15451 tablet.cc:1620] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:46.280046 15452 tablet.cc:1722] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:46.280337 15452 tablet.cc:1620] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:46.283659 9397 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:46.284605 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=9fd7fb73b74f43e1b15ded14256f18e4] alter complete (version 2)
I20240426 09:49:46.291837 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:46.293815 6993 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:46.294063 9527 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:46.295632 9660 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:46.297292 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:46.299686 9266 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:46.318845 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:46.330924 15391 consensus_queue.cc:1035] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:46.335211 15261 consensus_queue.cc:1035] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:46.370908 7274 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "643fd2d359a14f8eaa8d8a0d0879ef61"
dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:36536
I20240426 09:49:46.371094 7274 raft_consensus.cc:595] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:46.385982 15334 consensus_queue.cc:1035] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:46.388788 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:46.392369 13569 consensus_queue.cc:1035] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:46.394354 15224 tablet.cc:1722] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:46.399456 15451 tablet.cc:1722] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:46.401059 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=9fd7fb73b74f43e1b15ded14256f18e4] alter complete (version 3)
I20240426 09:49:46.404281 15452 tablet.cc:1722] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:46.472923 15403 consensus_queue.cc:1035] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:46.485522 14802 consensus_queue.cc:1035] T 5040b4c862a84a5786857fb0f53849ca P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:46.488595 6993 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:46.488664 9397 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:46.491396 15567 ts_tablet_manager.cc:902] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:46.492053 15567 tablet_copy_client.cc:308] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:46.493289 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 325e1d125617456f877204314d9e7bb9 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:41455)
I20240426 09:49:46.493638 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 325e1d125617456f877204314d9e7bb9 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:41455: session id = 558c4ecd0ece460d9591aa7a3c799d19-325e1d125617456f877204314d9e7bb9
I20240426 09:49:46.495011 7295 tablet_copy_source_session.cc:205] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:46.497735 9527 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:46.502171 9266 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:46.502931 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:46.502625 15567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 325e1d125617456f877204314d9e7bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.507660 6738 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:46.507802 6738 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:46.507869 15567 tablet_copy_client.cc:783] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:46.508044 15567 tablet_copy_client.cc:647] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:46.509176 15567 tablet_copy_client.cc:523] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:46.510818 15567 tablet_bootstrap.cc:492] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:46.516813 6738 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 06f9ef6c123f494bb8f72a034267fe29
I20240426 09:49:46.517494 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet 06f9ef6c123f494bb8f72a034267fe29 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:46 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:46.517750 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 06f9ef6c123f494bb8f72a034267fe29 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:46 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:46.518676 6860 tablet_service.cc:1514] Processing DeleteTablet for tablet 06f9ef6c123f494bb8f72a034267fe29 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:46 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:46.518802 15576 tablet_replica.cc:330] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:46.518951 15576 raft_consensus.cc:2229] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:46.519239 15576 raft_consensus.cc:2258] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:46.520023 15576 ts_tablet_manager.cc:1875] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:46.522864 15578 tablet_replica.cc:330] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:46.523222 15576 ts_tablet_manager.cc:1888] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:46.523367 15576 log.cc:1196] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/06f9ef6c123f494bb8f72a034267fe29
I20240426 09:49:46.523813 15576 ts_tablet_manager.cc:1909] T 06f9ef6c123f494bb8f72a034267fe29 P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:46.527580 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 06f9ef6c123f494bb8f72a034267fe29 (table rebalancer_dynamic_table_new_name [id=9fd7fb73b74f43e1b15ded14256f18e4]) successfully deleted
I20240426 09:49:46.530910 15578 raft_consensus.cc:2229] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:46.531131 15578 raft_consensus.cc:2258] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:46.534077 15578 ts_tablet_manager.cc:1875] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:46.536298 15578 ts_tablet_manager.cc:1888] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:46.536360 15578 log.cc:1196] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/06f9ef6c123f494bb8f72a034267fe29
I20240426 09:49:46.536654 15578 ts_tablet_manager.cc:1909] T 06f9ef6c123f494bb8f72a034267fe29 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:46.537353 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 06f9ef6c123f494bb8f72a034267fe29 (table rebalancer_dynamic_table_new_name [id=9fd7fb73b74f43e1b15ded14256f18e4]) successfully deleted
I20240426 09:49:46.552577 9660 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:46.559775 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:46.560598 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:46.562101 15391 raft_consensus.cc:979] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: : Instructing follower 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:49:46.562203 15261 raft_consensus.cc:1067] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Signalling peer 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:49:46.562748 7013 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "643fd2d359a14f8eaa8d8a0d0879ef61"
dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
from {username='slave'} at 127.0.99.132:32839
I20240426 09:49:46.562917 7013 raft_consensus.cc:482] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:46.562983 7013 raft_consensus.cc:3037] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:46.564358 7013 raft_consensus.cc:504] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.564893 7013 leader_election.cc:288] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:46.565728 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643fd2d359a14f8eaa8d8a0d0879ef61" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:46.565810 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643fd2d359a14f8eaa8d8a0d0879ef61" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:46.565851 7142 raft_consensus.cc:3037] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:46.565918 9672 raft_consensus.cc:3037] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:46.566857 7142 raft_consensus.cc:2454] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:49:46.566874 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "643fd2d359a14f8eaa8d8a0d0879ef61" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:46.566972 7274 raft_consensus.cc:3032] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:46.567011 7274 raft_consensus.cc:729] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:46.567154 7274 consensus_queue.cc:260] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.567329 7274 raft_consensus.cc:3037] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:46.568310 7274 raft_consensus.cc:2454] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:49:46.568693 6948 leader_election.cc:302] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:46.568873 14441 raft_consensus.cc:2781] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:46.568953 14441 raft_consensus.cc:686] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:46.569083 14441 consensus_queue.cc:237] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.570080 9672 raft_consensus.cc:2454] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:49:46.572036 6738 catalog_manager.cc:5484] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 1 to 2, leader changed from e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:46.574898 15577 tablet_replica.cc:330] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:46.575090 15577 raft_consensus.cc:2229] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:46.575337 15577 raft_consensus.cc:2258] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:46.576313 6736 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:46.582527 15577 ts_tablet_manager.cc:1875] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:46.589219 9527 tablet_service.cc:1467] Processing CreateTablet for tablet e6cb4534d05c49ed859b4abe61097ac8 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:46.589509 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6cb4534d05c49ed859b4abe61097ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.590286 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 11604d6436a946eba63a36c28e1b0290 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:46.590353 7255 tablet_service.cc:1467] Processing CreateTablet for tablet df622ccf230d49a58ce29308f3234690 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:46.590535 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11604d6436a946eba63a36c28e1b0290. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.590721 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df622ccf230d49a58ce29308f3234690. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.591564 9265 tablet_service.cc:1467] Processing CreateTablet for tablet df622ccf230d49a58ce29308f3234690 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:46.591842 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df622ccf230d49a58ce29308f3234690. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.593644 9266 tablet_service.cc:1467] Processing CreateTablet for tablet e6cb4534d05c49ed859b4abe61097ac8 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:46.593900 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6cb4534d05c49ed859b4abe61097ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.594599 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 11604d6436a946eba63a36c28e1b0290 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:46.594903 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11604d6436a946eba63a36c28e1b0290. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.597236 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 11604d6436a946eba63a36c28e1b0290 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:46.597339 15583 tablet_bootstrap.cc:492] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:46.597458 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11604d6436a946eba63a36c28e1b0290. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.598275 15583 tablet_bootstrap.cc:654] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.598667 9660 tablet_service.cc:1467] Processing CreateTablet for tablet e6cb4534d05c49ed859b4abe61097ac8 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:46.598959 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6cb4534d05c49ed859b4abe61097ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.601222 9661 tablet_service.cc:1467] Processing CreateTablet for tablet df622ccf230d49a58ce29308f3234690 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:46.601439 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df622ccf230d49a58ce29308f3234690. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.602267 15585 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:46.603156 15585 tablet_bootstrap.cc:654] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.603739 15577 ts_tablet_manager.cc:1888] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:46.604751 15577 log.cc:1196] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/06f9ef6c123f494bb8f72a034267fe29
I20240426 09:49:46.606500 15577 ts_tablet_manager.cc:1909] T 06f9ef6c123f494bb8f72a034267fe29 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:46.609311 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 06f9ef6c123f494bb8f72a034267fe29 (table rebalancer_dynamic_table_new_name [id=9fd7fb73b74f43e1b15ded14256f18e4]) successfully deleted
I20240426 09:49:46.609658 15580 tablet_bootstrap.cc:492] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:46.611001 15580 tablet_bootstrap.cc:654] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.616816 15585 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:46.616956 15585 ts_tablet_manager.cc:1367] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240426 09:49:46.617439 15585 raft_consensus.cc:348] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.617641 15585 raft_consensus.cc:374] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.618058 15585 raft_consensus.cc:729] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.618379 15585 consensus_queue.cc:260] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.618949 15585 ts_tablet_manager.cc:1398] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:46.619448 15585 tablet_bootstrap.cc:492] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:46.622373 6736 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:46.623221 15580 tablet_bootstrap.cc:492] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:46.623755 15580 ts_tablet_manager.cc:1367] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.001s
I20240426 09:49:46.624498 15580 raft_consensus.cc:348] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.624694 15580 raft_consensus.cc:374] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.624779 15580 raft_consensus.cc:729] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.624991 15580 consensus_queue.cc:260] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.627027 15580 ts_tablet_manager.cc:1398] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:49:46.628674 15580 tablet_bootstrap.cc:492] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:46.629480 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 3545007d995a4fb3beb8504710238682 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e4db1f2cdcdd4894ba2a7ba74a5931e3]), partition=
I20240426 09:49:46.629666 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 3545007d995a4fb3beb8504710238682 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e4db1f2cdcdd4894ba2a7ba74a5931e3]), partition=
I20240426 09:49:46.629715 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3545007d995a4fb3beb8504710238682. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.629918 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3545007d995a4fb3beb8504710238682. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.633399 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 3545007d995a4fb3beb8504710238682 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e4db1f2cdcdd4894ba2a7ba74a5931e3]), partition=
I20240426 09:49:46.633699 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3545007d995a4fb3beb8504710238682. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:46.638113 15585 tablet_bootstrap.cc:654] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.641167 15580 tablet_bootstrap.cc:654] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.646212 15588 tablet_bootstrap.cc:492] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:46.650156 15588 tablet_bootstrap.cc:654] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.659495 15588 tablet_bootstrap.cc:492] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:46.659690 15588 ts_tablet_manager.cc:1367] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.003s
I20240426 09:49:46.660856 15588 raft_consensus.cc:348] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:46.661634 15588 raft_consensus.cc:374] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.661772 15588 raft_consensus.cc:729] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.662457 15580 tablet_bootstrap.cc:492] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:46.662580 15580 ts_tablet_manager.cc:1367] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20240426 09:49:46.663261 15580 raft_consensus.cc:348] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.663389 15580 raft_consensus.cc:374] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.663431 15580 raft_consensus.cc:729] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.663611 15580 consensus_queue.cc:260] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.665006 15580 ts_tablet_manager.cc:1398] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:46.665246 15588 consensus_queue.cc:260] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:46.667232 15588 ts_tablet_manager.cc:1398] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20240426 09:49:46.673779 15581 tablet_bootstrap.cc:492] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:46.675578 15391 raft_consensus.cc:482] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:46.675690 15391 raft_consensus.cc:504] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.681248 15581 tablet_bootstrap.cc:654] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.681514 15582 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:46.686252 15567 tablet_bootstrap.cc:492] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:46.686612 15567 tablet_bootstrap.cc:492] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:46.686828 15567 ts_tablet_manager.cc:1367] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.176s user 0.046s sys 0.001s
I20240426 09:49:46.687474 15567 raft_consensus.cc:348] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.687682 15567 raft_consensus.cc:729] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.688412 15582 tablet_bootstrap.cc:654] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.689980 15391 leader_election.cc:288] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:46.690587 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df622ccf230d49a58ce29308f3234690" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:46.690882 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df622ccf230d49a58ce29308f3234690" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
W20240426 09:49:46.691269 7209 leader_election.cc:341] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Illegal state: must be running to vote when last-logged opid is not known
W20240426 09:49:46.691563 7208 leader_election.cc:341] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:46.691619 7208 leader_election.cc:302] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e3394a4be9ea4def8aa39b333c82c5ff; no voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:46.691876 15391 raft_consensus.cc:2726] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240426 09:49:46.690893 15567 consensus_queue.cc:260] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.692631 15567 ts_tablet_manager.cc:1398] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20240426 09:49:46.693148 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-325e1d125617456f877204314d9e7bb9 received from {username='slave'} at 127.0.99.136:41455
I20240426 09:49:46.693225 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-325e1d125617456f877204314d9e7bb9 on tablet 325e1d125617456f877204314d9e7bb9 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:46.696640 15585 tablet_bootstrap.cc:492] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:46.696717 15585 ts_tablet_manager.cc:1367] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.077s user 0.002s sys 0.000s
I20240426 09:49:46.697116 15585 raft_consensus.cc:348] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.697211 15585 raft_consensus.cc:374] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.697237 15585 raft_consensus.cc:729] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.697336 15585 consensus_queue.cc:260] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.697603 15585 ts_tablet_manager.cc:1398] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:46.697772 15585 tablet_bootstrap.cc:492] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:46.698611 15585 tablet_bootstrap.cc:654] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.699589 15581 tablet_bootstrap.cc:492] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:46.699700 15581 ts_tablet_manager.cc:1367] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240426 09:49:46.701862 15581 raft_consensus.cc:348] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.702159 15581 raft_consensus.cc:374] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.702244 15581 raft_consensus.cc:729] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.702656 15581 consensus_queue.cc:260] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.703858 15581 ts_tablet_manager.cc:1398] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:46.704173 15581 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:46.706969 15581 tablet_bootstrap.cc:654] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.708992 15582 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:46.709124 15582 ts_tablet_manager.cc:1367] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20240426 09:49:46.709575 15582 raft_consensus.cc:348] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.709750 15582 raft_consensus.cc:374] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.709813 15582 raft_consensus.cc:729] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.710011 15582 consensus_queue.cc:260] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.711212 15582 ts_tablet_manager.cc:1398] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:46.711431 15582 tablet_bootstrap.cc:492] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:46.713187 15582 tablet_bootstrap.cc:654] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.725225 15583 tablet_bootstrap.cc:492] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:46.725351 15583 ts_tablet_manager.cc:1367] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.128s user 0.002s sys 0.000s
I20240426 09:49:46.725795 15583 raft_consensus.cc:348] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.725893 15583 raft_consensus.cc:374] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.725919 15583 raft_consensus.cc:729] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.726040 15583 consensus_queue.cc:260] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.726292 15583 ts_tablet_manager.cc:1398] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:46.726451 15583 tablet_bootstrap.cc:492] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:46.727734 15585 tablet_bootstrap.cc:492] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:46.727838 15585 ts_tablet_manager.cc:1367] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20240426 09:49:46.728315 15585 raft_consensus.cc:348] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:46.728425 15585 raft_consensus.cc:374] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.728454 15585 raft_consensus.cc:729] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.728565 15585 consensus_queue.cc:260] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:46.728843 15585 ts_tablet_manager.cc:1398] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:46.732417 15583 tablet_bootstrap.cc:654] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:46.733493 15583 tablet_bootstrap.cc:492] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:46.733922 15583 ts_tablet_manager.cc:1367] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:46.734431 15583 raft_consensus.cc:348] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:46.734651 15583 raft_consensus.cc:374] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.734711 15583 raft_consensus.cc:729] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.734879 15583 consensus_queue.cc:260] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:46.735190 15583 ts_tablet_manager.cc:1398] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:46.735747 15581 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:46.735852 15581 ts_tablet_manager.cc:1367] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240426 09:49:46.736234 15582 tablet_bootstrap.cc:492] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:46.736331 15582 ts_tablet_manager.cc:1367] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240426 09:49:46.736805 15582 raft_consensus.cc:348] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.736927 15582 raft_consensus.cc:374] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.736956 15582 raft_consensus.cc:729] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.737078 15582 consensus_queue.cc:260] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.737340 15582 ts_tablet_manager.cc:1398] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:46.737864 15581 raft_consensus.cc:348] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.737974 15581 raft_consensus.cc:374] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:46.737995 15581 raft_consensus.cc:729] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:46.738111 15581 consensus_queue.cc:260] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.738359 15581 ts_tablet_manager.cc:1398] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240426 09:49:46.867566 14799 raft_consensus.cc:482] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:46.867699 14799 raft_consensus.cc:504] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.868124 14799 leader_election.cc:288] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:46.868523 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6cb4534d05c49ed859b4abe61097ac8" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:46.868666 9672 raft_consensus.cc:2454] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:46.868940 9219 leader_election.cc:302] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:46.868970 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6cb4534d05c49ed859b4abe61097ac8" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:46.869086 9549 raft_consensus.cc:2454] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:46.869470 14799 raft_consensus.cc:2781] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:46.869527 14799 raft_consensus.cc:482] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:46.869572 14799 raft_consensus.cc:3037] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.870549 14799 raft_consensus.cc:504] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.870864 14799 leader_election.cc:288] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:46.871186 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6cb4534d05c49ed859b4abe61097ac8" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:46.871291 9672 raft_consensus.cc:3037] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.872323 9672 raft_consensus.cc:2454] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:46.872550 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6cb4534d05c49ed859b4abe61097ac8" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:46.872607 9219 leader_election.cc:302] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:46.872686 9549 raft_consensus.cc:3037] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.872761 14799 raft_consensus.cc:2781] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:46.872848 14799 raft_consensus.cc:686] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:46.872996 14799 consensus_queue.cc:237] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:46.873672 9549 raft_consensus.cc:2454] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:46.875272 6738 catalog_manager.cc:5484] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:46.925266 14361 raft_consensus.cc:482] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:46.925386 14361 raft_consensus.cc:504] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.925835 14361 leader_election.cc:288] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:46.926151 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df622ccf230d49a58ce29308f3234690" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:46.926280 7274 raft_consensus.cc:2454] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:46.926403 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df622ccf230d49a58ce29308f3234690" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:46.926542 9285 raft_consensus.cc:2454] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:46.926538 9616 leader_election.cc:302] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:46.926622 15391 raft_consensus.cc:482] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:46.926713 15391 raft_consensus.cc:504] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.927048 14361 raft_consensus.cc:2781] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:46.927103 14361 raft_consensus.cc:482] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:46.927139 14361 raft_consensus.cc:3037] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.927187 15391 leader_election.cc:288] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:46.927181 14802 raft_consensus.cc:482] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:46.927263 14802 raft_consensus.cc:504] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.927619 14802 leader_election.cc:288] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:46.927989 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11604d6436a946eba63a36c28e1b0290" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:46.928103 7013 raft_consensus.cc:2454] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:46.928128 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11604d6436a946eba63a36c28e1b0290" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:46.928242 7011 raft_consensus.cc:2454] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:46.928216 14361 raft_consensus.cc:504] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.928359 7208 leader_election.cc:302] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:46.928468 9482 leader_election.cc:302] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:46.928524 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11604d6436a946eba63a36c28e1b0290" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:46.928548 14361 leader_election.cc:288] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:46.928571 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11604d6436a946eba63a36c28e1b0290" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:46.928620 7274 raft_consensus.cc:2454] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:46.928669 9547 raft_consensus.cc:2454] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:46.928683 15391 raft_consensus.cc:2781] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:46.928745 15391 raft_consensus.cc:482] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:46.928786 15391 raft_consensus.cc:3037] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.928879 14802 raft_consensus.cc:2781] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:46.928925 14802 raft_consensus.cc:482] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:46.928965 14802 raft_consensus.cc:3037] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.929481 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df622ccf230d49a58ce29308f3234690" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:46.929589 9285 raft_consensus.cc:3037] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.929878 14802 raft_consensus.cc:504] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.929878 15391 raft_consensus.cc:504] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.930169 15391 leader_election.cc:288] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:46.930184 14802 leader_election.cc:288] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:46.930397 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11604d6436a946eba63a36c28e1b0290" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:46.930516 7275 raft_consensus.cc:2379] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in current term 1: Already voted for candidate e3394a4be9ea4def8aa39b333c82c5ff in this term.
I20240426 09:49:46.930547 9285 raft_consensus.cc:2454] T df622ccf230d49a58ce29308f3234690 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:46.931080 9615 leader_election.cc:302] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:46.931305 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df622ccf230d49a58ce29308f3234690" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:46.931401 7274 raft_consensus.cc:3037] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.931860 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11604d6436a946eba63a36c28e1b0290" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:46.931957 7011 raft_consensus.cc:3037] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:46.932262 7274 raft_consensus.cc:2454] T df622ccf230d49a58ce29308f3234690 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:46.932963 7011 raft_consensus.cc:2454] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:46.933126 14361 raft_consensus.cc:2781] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:46.933207 14361 raft_consensus.cc:686] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:46.933352 14361 consensus_queue.cc:237] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:46.933707 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11604d6436a946eba63a36c28e1b0290" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:46.933825 9547 raft_consensus.cc:2379] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e3394a4be9ea4def8aa39b333c82c5ff in current term 1: Already voted for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in this term.
I20240426 09:49:46.934188 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11604d6436a946eba63a36c28e1b0290" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:46.934319 7011 raft_consensus.cc:2379] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e3394a4be9ea4def8aa39b333c82c5ff in current term 1: Already voted for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in this term.
I20240426 09:49:46.934849 7208 leader_election.cc:302] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e3394a4be9ea4def8aa39b333c82c5ff; no voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:46.935070 15391 raft_consensus.cc:2726] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240426 09:49:46.935918 6735 catalog_manager.cc:5484] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:46.936730 9482 leader_election.cc:302] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters: e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:46.937417 14802 raft_consensus.cc:2781] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:46.937494 14802 raft_consensus.cc:686] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:46.937603 14802 consensus_queue.cc:237] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:46.939576 6735 catalog_manager.cc:5484] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:47.039664 9672 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:47.040066 14441 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:47.058404 9672 raft_consensus.cc:1203] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:47.066080 15450 raft_consensus.cc:482] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:47.066201 15450 raft_consensus.cc:504] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.066519 15450 leader_election.cc:288] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:47.066890 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3545007d995a4fb3beb8504710238682" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:47.066912 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3545007d995a4fb3beb8504710238682" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:47.067021 7011 raft_consensus.cc:2454] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:47.067040 9672 raft_consensus.cc:2454] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:47.067317 7077 leader_election.cc:302] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:47.067495 15450 raft_consensus.cc:2781] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:47.067559 15450 raft_consensus.cc:482] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:47.067615 15450 raft_consensus.cc:3037] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.068763 15450 raft_consensus.cc:504] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.069067 15450 leader_election.cc:288] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:47.069339 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3545007d995a4fb3beb8504710238682" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:47.069483 7011 raft_consensus.cc:3037] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.069447 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3545007d995a4fb3beb8504710238682" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:47.069572 9672 raft_consensus.cc:3037] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.070261 7011 raft_consensus.cc:2454] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:47.070408 9672 raft_consensus.cc:2454] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:47.070499 7077 leader_election.cc:302] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:47.070789 15450 raft_consensus.cc:2781] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:47.070964 15450 raft_consensus.cc:686] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:47.071300 15450 consensus_queue.cc:237] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.073422 6735 catalog_manager.cc:5484] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:47.090037 6993 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:47.092485 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:47.101625 9397 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:47.102108 9661 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:47.102444 9529 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:47.105789 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:47.111665 9266 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:47.121874 7253 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:47.161760 7142 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:47.162169 14441 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:47.164804 7274 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:49:47.165481 14441 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:47.186425 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:47.191125 7011 raft_consensus.cc:1261] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:47.191126 9672 raft_consensus.cc:1261] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:47.191542 15570 consensus_queue.cc:1035] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.191817 15450 consensus_queue.cc:1035] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.192220 7011 consensus_queue.cc:237] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.194159 9285 raft_consensus.cc:1261] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:47.194646 14441 consensus_queue.cc:1035] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:47.195232 9669 raft_consensus.cc:1261] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:47.195732 6735 catalog_manager.cc:5576] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135) does not have the latest schema for tablet 3545007d995a4fb3beb8504710238682 (table rebalancer_dynamic_table [id=e4db1f2cdcdd4894ba2a7ba74a5931e3]). Expected version 1 got 0
I20240426 09:49:47.196053 14441 consensus_queue.cc:1035] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:47.196858 15689 tablet.cc:1722] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:47.197183 6735 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet 3545007d995a4fb3beb8504710238682 (table rebalancer_dynamic_table [id=e4db1f2cdcdd4894ba2a7ba74a5931e3]). Expected version 1 got 0
I20240426 09:49:47.197310 15689 tablet.cc:1620] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:47.199189 14441 raft_consensus.cc:2932] T 852fad44c59f42dd9ce66b84819a9296 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:47.199555 9669 raft_consensus.cc:2932] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:47.201943 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=e4db1f2cdcdd4894ba2a7ba74a5931e3] alter complete (version 1)
I20240426 09:49:47.204572 6735 catalog_manager.cc:5484] T 852fad44c59f42dd9ce66b84819a9296 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:47.206914 15688 tablet.cc:1722] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:47.207273 15688 tablet.cc:1620] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:47.207928 15698 tablet.cc:1722] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:47.208230 15698 tablet.cc:1620] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:47.208555 9285 raft_consensus.cc:2932] T 852fad44c59f42dd9ce66b84819a9296 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:47.212149 9529 tablet_service.cc:1514] Processing DeleteTablet for tablet 852fad44c59f42dd9ce66b84819a9296 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:47.212539 15699 tablet_replica.cc:330] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:47.212874 15699 raft_consensus.cc:2229] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:47.213080 15699 raft_consensus.cc:2258] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:47.213608 15699 ts_tablet_manager.cc:1875] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:47.215866 15699 ts_tablet_manager.cc:1888] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:47.215989 15699 log.cc:1196] T 852fad44c59f42dd9ce66b84819a9296 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/852fad44c59f42dd9ce66b84819a9296
I20240426 09:49:47.216558 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 852fad44c59f42dd9ce66b84819a9296 (table rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]) successfully deleted
I20240426 09:49:47.306715 6731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:47.316325 15688 tablet.cc:1722] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:47.316659 15688 tablet.cc:1620] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:47.316671 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:47.316859 15689 tablet.cc:1722] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:47.317160 15689 tablet.cc:1620] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:47.320761 6993 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:47.321007 9266 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:47.327817 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=e4db1f2cdcdd4894ba2a7ba74a5931e3] alter complete (version 2)
I20240426 09:49:47.327809 15698 tablet.cc:1722] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:47.328083 15698 tablet.cc:1620] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:47.329033 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:47.329277 9661 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:47.332015 9397 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240426 09:49:47.333921 14361 consensus_queue.cc:1035] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.337348 15494 consensus_queue.cc:1035] T df622ccf230d49a58ce29308f3234690 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.372295 9529 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:47.372866 7253 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:47.382256 14802 consensus_queue.cc:1035] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.392937 14802 consensus_queue.cc:1035] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.398630 14799 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.410735 15309 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.433312 6731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:47.438999 15688 tablet.cc:1722] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:47.439653 15689 tablet.cc:1722] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:47.439725 15698 tablet.cc:1722] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:47.444903 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=e4db1f2cdcdd4894ba2a7ba74a5931e3] alter complete (version 3)
I20240426 09:49:47.450059 7142 consensus_queue.cc:237] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:47.451372 9547 raft_consensus.cc:1261] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:47.451387 6882 raft_consensus.cc:1261] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:47.451757 15450 consensus_queue.cc:1035] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:47.452132 15766 consensus_queue.cc:1035] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:47.454075 15766 raft_consensus.cc:2932] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:47.454571 6882 raft_consensus.cc:2932] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:47.455076 9547 raft_consensus.cc:2932] T 639d1d6189fb4d5f8a2d0aa0d39575ec P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:47.457715 6731 catalog_manager.cc:5484] T 639d1d6189fb4d5f8a2d0aa0d39575ec P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:47.461120 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 639d1d6189fb4d5f8a2d0aa0d39575ec with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:47.461462 15780 tablet_replica.cc:330] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:47.461611 15780 raft_consensus.cc:2229] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:47.461756 15780 raft_consensus.cc:2258] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:47.462272 15780 ts_tablet_manager.cc:1875] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:47.465150 15780 ts_tablet_manager.cc:1888] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:47.465232 15780 log.cc:1196] T 639d1d6189fb4d5f8a2d0aa0d39575ec P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/639d1d6189fb4d5f8a2d0aa0d39575ec
I20240426 09:49:47.465791 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 639d1d6189fb4d5f8a2d0aa0d39575ec (table rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]) successfully deleted
I20240426 09:49:47.507803 6735 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:47.513985 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 638b15a2c9ea42c3b86a985ccae550a7 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:47.514304 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638b15a2c9ea42c3b86a985ccae550a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.514658 9265 tablet_service.cc:1467] Processing CreateTablet for tablet d2a5223682934aa698dd23aa6eab941b (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:47.515108 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2a5223682934aa698dd23aa6eab941b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.515311 6860 tablet_service.cc:1467] Processing CreateTablet for tablet d2a5223682934aa698dd23aa6eab941b (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:47.515784 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2a5223682934aa698dd23aa6eab941b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.516693 15821 tablet_bootstrap.cc:492] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:47.516844 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 638b15a2c9ea42c3b86a985ccae550a7 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:47.517071 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638b15a2c9ea42c3b86a985ccae550a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.517956 15821 tablet_bootstrap.cc:654] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.519194 15821 tablet_bootstrap.cc:492] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:47.519277 15821 ts_tablet_manager.cc:1367] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:47.519734 15821 raft_consensus.cc:348] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.519837 15821 raft_consensus.cc:374] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.519878 15821 raft_consensus.cc:729] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.519979 15821 consensus_queue.cc:260] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.520255 15821 ts_tablet_manager.cc:1398] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:47.520570 9660 tablet_service.cc:1467] Processing CreateTablet for tablet c10b7673a7bf4980add3d94c6284e57b (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:47.520710 9395 tablet_service.cc:1467] Processing CreateTablet for tablet c10b7673a7bf4980add3d94c6284e57b (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:47.520831 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c10b7673a7bf4980add3d94c6284e57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.520800 9397 tablet_service.cc:1467] Processing CreateTablet for tablet d2a5223682934aa698dd23aa6eab941b (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:47.520931 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c10b7673a7bf4980add3d94c6284e57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.520586 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 638b15a2c9ea42c3b86a985ccae550a7 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:47.521219 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638b15a2c9ea42c3b86a985ccae550a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.521704 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2a5223682934aa698dd23aa6eab941b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.524844 7122 tablet_service.cc:1467] Processing CreateTablet for tablet c10b7673a7bf4980add3d94c6284e57b (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:47.525321 15822 tablet_bootstrap.cc:492] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:47.526631 15822 tablet_bootstrap.cc:654] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.526933 15823 tablet_bootstrap.cc:492] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:47.527397 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c10b7673a7bf4980add3d94c6284e57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.527833 15822 tablet_bootstrap.cc:492] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:47.527912 15822 ts_tablet_manager.cc:1367] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240426 09:49:47.527969 15826 tablet_bootstrap.cc:492] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:47.528347 15822 raft_consensus.cc:348] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.528472 15822 raft_consensus.cc:374] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.528502 15822 raft_consensus.cc:729] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.528613 15822 consensus_queue.cc:260] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.528919 15826 tablet_bootstrap.cc:654] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.528940 15822 ts_tablet_manager.cc:1398] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:47.529114 15822 tablet_bootstrap.cc:492] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:47.529664 15823 tablet_bootstrap.cc:654] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.529963 15822 tablet_bootstrap.cc:654] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.530577 15821 tablet_bootstrap.cc:492] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:47.533916 15822 tablet_bootstrap.cc:492] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:47.533938 15821 tablet_bootstrap.cc:654] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.534004 15822 ts_tablet_manager.cc:1367] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240426 09:49:47.534382 15822 raft_consensus.cc:348] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.534587 15822 raft_consensus.cc:374] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.534683 15822 raft_consensus.cc:729] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.534870 15822 consensus_queue.cc:260] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.535023 15821 tablet_bootstrap.cc:492] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:47.535102 15821 ts_tablet_manager.cc:1367] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:47.535205 15822 ts_tablet_manager.cc:1398] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:47.535468 15821 raft_consensus.cc:348] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.535557 15821 raft_consensus.cc:374] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.535583 15821 raft_consensus.cc:729] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.535679 15821 consensus_queue.cc:260] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.535872 15826 tablet_bootstrap.cc:492] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:47.535897 15821 ts_tablet_manager.cc:1398] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:47.535957 15826 ts_tablet_manager.cc:1367] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:49:47.536442 15826 raft_consensus.cc:348] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.536546 15826 raft_consensus.cc:374] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.536612 15826 raft_consensus.cc:729] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.536813 15825 tablet_bootstrap.cc:492] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:47.536782 15826 consensus_queue.cc:260] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.537233 15826 ts_tablet_manager.cc:1398] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:47.537420 15826 tablet_bootstrap.cc:492] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:47.538005 15825 tablet_bootstrap.cc:654] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.539280 15826 tablet_bootstrap.cc:654] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.540350 15826 tablet_bootstrap.cc:492] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:47.540432 15826 ts_tablet_manager.cc:1367] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20240426 09:49:47.540840 15826 raft_consensus.cc:348] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.540931 15826 raft_consensus.cc:374] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.540964 15826 raft_consensus.cc:729] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.541035 15825 tablet_bootstrap.cc:492] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:47.541066 15826 consensus_queue.cc:260] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.541127 15825 ts_tablet_manager.cc:1367] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:47.541325 15826 ts_tablet_manager.cc:1398] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:47.541630 15825 raft_consensus.cc:348] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.541838 15825 raft_consensus.cc:374] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.541913 15825 raft_consensus.cc:729] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.542071 15825 consensus_queue.cc:260] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.542426 15825 ts_tablet_manager.cc:1398] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:47.543349 15825 tablet_bootstrap.cc:492] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:47.544270 15825 tablet_bootstrap.cc:654] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.546175 15825 tablet_bootstrap.cc:492] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:47.546303 15825 ts_tablet_manager.cc:1367] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:47.546711 15825 raft_consensus.cc:348] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.546831 15825 raft_consensus.cc:374] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.546896 15825 raft_consensus.cc:729] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.547016 15825 consensus_queue.cc:260] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.547309 15825 ts_tablet_manager.cc:1398] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:47.547443 15823 tablet_bootstrap.cc:492] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:47.547561 15823 ts_tablet_manager.cc:1367] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240426 09:49:47.547987 15823 raft_consensus.cc:348] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.548081 15823 raft_consensus.cc:374] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.548113 15823 raft_consensus.cc:729] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.548240 15823 consensus_queue.cc:260] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.548516 15823 ts_tablet_manager.cc:1398] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:47.549439 6731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:47.549562 6731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:47.552106 9672 consensus_queue.cc:237] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:47.553982 9285 raft_consensus.cc:1261] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:47.554111 9616 consensus_peers.cc:485] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2c41aab70e4a4b4e86cf095575cd8805. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:47.554585 15841 consensus_queue.cc:1035] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:47.556205 9547 raft_consensus.cc:1261] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:47.556911 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 3545007d995a4fb3beb8504710238682
I20240426 09:49:47.556883 15841 consensus_queue.cc:1035] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:47.557649 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 3545007d995a4fb3beb8504710238682 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:47 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:47.557636 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 3545007d995a4fb3beb8504710238682 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:47 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:47.557785 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 3545007d995a4fb3beb8504710238682 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:47 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:47.557823 15780 tablet_replica.cc:330] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:47.557956 15780 raft_consensus.cc:2229] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:47.558077 15780 raft_consensus.cc:2258] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:47.558794 15780 ts_tablet_manager.cc:1875] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:47.559489 15844 tablet_replica.cc:330] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:47.559615 15844 raft_consensus.cc:2229] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:47.559813 15844 raft_consensus.cc:2258] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:47.561196 15780 ts_tablet_manager.cc:1888] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:47.561275 15780 log.cc:1196] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/3545007d995a4fb3beb8504710238682
I20240426 09:49:47.561540 15780 ts_tablet_manager.cc:1909] T 3545007d995a4fb3beb8504710238682 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:47.561990 15844 ts_tablet_manager.cc:1875] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:47.562228 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 3545007d995a4fb3beb8504710238682 (table rebalancer_dynamic_table_new_name [id=e4db1f2cdcdd4894ba2a7ba74a5931e3]) successfully deleted
I20240426 09:49:47.563169 15843 tablet_replica.cc:330] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:47.563313 15843 raft_consensus.cc:2229] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:47.563437 15843 raft_consensus.cc:2258] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:47.563952 15843 ts_tablet_manager.cc:1875] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:47.564194 15844 ts_tablet_manager.cc:1888] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:47.564316 15844 log.cc:1196] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/3545007d995a4fb3beb8504710238682
I20240426 09:49:47.564658 15844 ts_tablet_manager.cc:1909] T 3545007d995a4fb3beb8504710238682 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:47.565368 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 3545007d995a4fb3beb8504710238682 (table rebalancer_dynamic_table_new_name [id=e4db1f2cdcdd4894ba2a7ba74a5931e3]) successfully deleted
I20240426 09:49:47.567037 15843 ts_tablet_manager.cc:1888] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:47.567184 15843 log.cc:1196] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/3545007d995a4fb3beb8504710238682
I20240426 09:49:47.567525 15843 ts_tablet_manager.cc:1909] T 3545007d995a4fb3beb8504710238682 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:47.567618 15841 raft_consensus.cc:2932] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:47.569355 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 3545007d995a4fb3beb8504710238682 (table rebalancer_dynamic_table_new_name [id=e4db1f2cdcdd4894ba2a7ba74a5931e3]) successfully deleted
I20240426 09:49:47.569759 9547 raft_consensus.cc:2932] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:47.571910 6735 catalog_manager.cc:5484] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:47.571936 9285 raft_consensus.cc:2932] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:47.593189 7142 consensus_queue.cc:237] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.594494 7013 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:47.594549 7274 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:47.594945 7077 consensus_peers.cc:485] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd377032e7cd4b4998424f8685ad5b85. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:47.594988 15766 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:47.595175 15450 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:47.597901 15766 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:47.598353 7013 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:47.599694 7274 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:47.600666 6731 catalog_manager.cc:5484] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:47.621179 6882 consensus_queue.cc:237] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.622696 7274 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:47.622718 6815 consensus_peers.cc:485] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5fe43b0765d9401a88c8e9a028ab3ef4. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:47.622843 7142 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:47.623080 15178 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:47.623468 15178 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:47.625016 15882 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:47.625419 7142 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:47.627262 7274 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:47.628748 6735 catalog_manager.cc:5484] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:47.650647 6882 consensus_queue.cc:237] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:47.652191 7013 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:47.652268 7142 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:47.652436 6815 consensus_peers.cc:485] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1fa5a98645834e2886ff45b8b28c1c91. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:47.652621 15178 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:47.652797 15882 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:47.654631 15882 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:47.655020 7013 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:47.656777 7142 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:47.657999 6737 catalog_manager.cc:5484] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:47.662683 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:47.668253 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 82c3f4fc1e304580a287c0fc031614db (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e9ddab232c634786a34bc8ea3b449f19]), partition=
I20240426 09:49:47.668475 14361 raft_consensus.cc:482] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:47.668524 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 82c3f4fc1e304580a287c0fc031614db (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e9ddab232c634786a34bc8ea3b449f19]), partition=
I20240426 09:49:47.668560 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c3f4fc1e304580a287c0fc031614db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.668576 14361 raft_consensus.cc:504] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.668726 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c3f4fc1e304580a287c0fc031614db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.669075 14361 leader_election.cc:288] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:47.669536 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c10b7673a7bf4980add3d94c6284e57b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:47.669658 9417 raft_consensus.cc:2454] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:47.669734 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 82c3f4fc1e304580a287c0fc031614db (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e9ddab232c634786a34bc8ea3b449f19]), partition=
I20240426 09:49:47.669952 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c3f4fc1e304580a287c0fc031614db. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:47.670679 15823 tablet_bootstrap.cc:492] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:47.671640 15823 tablet_bootstrap.cc:654] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.672609 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c10b7673a7bf4980add3d94c6284e57b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:47.672744 7143 raft_consensus.cc:2454] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:47.672821 9614 leader_election.cc:302] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:47.673259 15494 raft_consensus.cc:2781] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:47.673317 15494 raft_consensus.cc:482] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:47.673358 15494 raft_consensus.cc:3037] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.674166 15916 tablet_bootstrap.cc:492] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:47.674388 15494 raft_consensus.cc:504] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.674710 15494 leader_election.cc:288] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:47.675115 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c10b7673a7bf4980add3d94c6284e57b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:47.675158 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c10b7673a7bf4980add3d94c6284e57b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:47.675230 9417 raft_consensus.cc:3037] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.675249 7143 raft_consensus.cc:3037] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.676157 9417 raft_consensus.cc:2454] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:47.676157 7143 raft_consensus.cc:2454] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:47.676394 15823 tablet_bootstrap.cc:492] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:47.676520 15823 ts_tablet_manager.cc:1367] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:47.676751 9614 leader_election.cc:302] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:47.676896 15494 raft_consensus.cc:2781] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:47.676961 15494 raft_consensus.cc:686] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:47.676966 15823 raft_consensus.cc:348] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.677057 15823 raft_consensus.cc:374] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.677083 15823 raft_consensus.cc:729] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.677101 15494 consensus_queue.cc:237] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:47.677196 15823 consensus_queue.cc:260] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.677469 15823 ts_tablet_manager.cc:1398] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:47.679838 6738 catalog_manager.cc:5484] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:47.680963 15916 tablet_bootstrap.cc:654] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.681638 15909 tablet_bootstrap.cc:492] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:47.682696 15909 tablet_bootstrap.cc:654] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:47.683516 15595 raft_consensus.cc:482] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:47.683610 15595 raft_consensus.cc:504] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.683964 15595 leader_election.cc:288] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:47.684437 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3f4fc1e304580a287c0fc031614db" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:47.684913 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3f4fc1e304580a287c0fc031614db" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
W20240426 09:49:47.685315 6815 leader_election.cc:341] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Illegal state: must be running to vote when last-logged opid is not known
W20240426 09:49:47.685428 6815 leader_election.cc:341] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:47.685464 6815 leader_election.cc:302] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c; no voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:47.685572 15595 raft_consensus.cc:2726] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240426 09:49:47.687492 15909 tablet_bootstrap.cc:492] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:47.687516 15916 tablet_bootstrap.cc:492] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:47.687580 15909 ts_tablet_manager.cc:1367] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:47.687598 15916 ts_tablet_manager.cc:1367] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:47.688534 15909 raft_consensus.cc:348] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.688647 15909 raft_consensus.cc:374] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.688589 15916 raft_consensus.cc:348] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.688683 15909 raft_consensus.cc:729] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.688690 15916 raft_consensus.cc:374] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:47.688725 15916 raft_consensus.cc:729] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:47.688793 15909 consensus_queue.cc:260] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.688859 15916 consensus_queue.cc:260] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.689139 15916 ts_tablet_manager.cc:1398] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:47.689253 15909 ts_tablet_manager.cc:1398] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:47.726346 15691 raft_consensus.cc:482] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:47.726470 15691 raft_consensus.cc:504] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.726922 15691 leader_election.cc:288] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:47.727243 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2a5223682934aa698dd23aa6eab941b" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:47.727314 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2a5223682934aa698dd23aa6eab941b" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:47.727416 6882 raft_consensus.cc:2454] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:47.727494 9285 raft_consensus.cc:2454] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:47.727782 9351 leader_election.cc:302] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:47.727944 15691 raft_consensus.cc:2781] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:47.728008 15691 raft_consensus.cc:482] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:47.728050 15691 raft_consensus.cc:3037] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.729228 15691 raft_consensus.cc:504] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.729645 15691 leader_election.cc:288] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:47.729888 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2a5223682934aa698dd23aa6eab941b" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:47.730005 6882 raft_consensus.cc:3037] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.730574 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2a5223682934aa698dd23aa6eab941b" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:47.730674 9285 raft_consensus.cc:3037] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.731379 6882 raft_consensus.cc:2454] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:47.731652 9285 raft_consensus.cc:2454] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:47.731966 9352 leader_election.cc:302] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:47.732410 15691 raft_consensus.cc:2781] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:47.732579 15691 raft_consensus.cc:686] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:47.732730 15691 consensus_queue.cc:237] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:47.734998 6736 catalog_manager.cc:5484] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:47.738307 15309 raft_consensus.cc:482] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:47.738425 15309 raft_consensus.cc:504] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.738977 15309 leader_election.cc:288] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:47.739430 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638b15a2c9ea42c3b86a985ccae550a7" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:47.739415 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638b15a2c9ea42c3b86a985ccae550a7" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:47.739545 7143 raft_consensus.cc:2454] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:47.739545 9672 raft_consensus.cc:2454] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:47.739835 9219 leader_election.cc:302] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:47.740123 15309 raft_consensus.cc:2781] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:47.740178 15309 raft_consensus.cc:482] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:47.740219 15309 raft_consensus.cc:3037] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.741295 15309 raft_consensus.cc:504] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.741606 15309 leader_election.cc:288] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:47.742138 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638b15a2c9ea42c3b86a985ccae550a7" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:47.742161 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "638b15a2c9ea42c3b86a985ccae550a7" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:47.742252 9672 raft_consensus.cc:3037] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.742269 7143 raft_consensus.cc:3037] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.743360 9672 raft_consensus.cc:2454] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:47.743436 7143 raft_consensus.cc:2454] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:47.743798 9219 leader_election.cc:302] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:47.744022 15309 raft_consensus.cc:2781] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:47.744099 15309 raft_consensus.cc:686] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:47.744226 15309 consensus_queue.cc:237] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:47.751490 6736 catalog_manager.cc:5484] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:47.835891 15670 raft_consensus.cc:482] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:47.835994 15670 raft_consensus.cc:504] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.836470 15670 leader_election.cc:288] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:47.836782 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3f4fc1e304580a287c0fc031614db" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:47.836831 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3f4fc1e304580a287c0fc031614db" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:47.836928 6880 raft_consensus.cc:2454] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:47.836957 9549 raft_consensus.cc:2454] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:47.837172 6946 leader_election.cc:302] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:47.837329 15670 raft_consensus.cc:2781] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:47.837383 15670 raft_consensus.cc:482] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:47.837415 15670 raft_consensus.cc:3037] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.838280 15670 raft_consensus.cc:504] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.838631 15670 leader_election.cc:288] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:47.839054 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3f4fc1e304580a287c0fc031614db" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:47.839166 6880 raft_consensus.cc:3037] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.840070 6880 raft_consensus.cc:2454] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:47.840292 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c3f4fc1e304580a287c0fc031614db" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:47.840320 6948 leader_election.cc:302] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:47.840413 9549 raft_consensus.cc:3037] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:47.840477 15670 raft_consensus.cc:2781] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:47.840546 15670 raft_consensus.cc:686] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:47.840677 15670 consensus_queue.cc:237] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:47.841318 9549 raft_consensus.cc:2454] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:47.842689 6738 catalog_manager.cc:5484] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:47.860867 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:47.866567 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:47.867599 9661 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:47.872196 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:47.873442 7253 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:47.873824 9395 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:47.907703 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:47.922217 9529 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:47.980756 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:47.985261 9549 raft_consensus.cc:1261] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:47.985260 6882 raft_consensus.cc:1261] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:47.985970 15237 consensus_queue.cc:1035] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.986100 15670 consensus_queue.cc:1035] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:47.989081 6738 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet 82c3f4fc1e304580a287c0fc031614db (table rebalancer_dynamic_table [id=e9ddab232c634786a34bc8ea3b449f19]). Expected version 1 got 0
I20240426 09:49:47.989574 6739 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 82c3f4fc1e304580a287c0fc031614db (table rebalancer_dynamic_table [id=e9ddab232c634786a34bc8ea3b449f19]). Expected version 1 got 0
I20240426 09:49:47.990814 15994 tablet.cc:1722] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:47.991393 15994 tablet.cc:1620] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:47.992295 6739 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet 82c3f4fc1e304580a287c0fc031614db (table rebalancer_dynamic_table [id=e9ddab232c634786a34bc8ea3b449f19]). Expected version 1 got 0
I20240426 09:49:47.989804 15995 tablet.cc:1722] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:47.992702 15995 tablet.cc:1620] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:47.993394 6739 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 82c3f4fc1e304580a287c0fc031614db (table rebalancer_dynamic_table [id=e9ddab232c634786a34bc8ea3b449f19]). Expected version 1 got 0
I20240426 09:49:47.995081 15996 tablet.cc:1722] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:47.995375 15996 tablet.cc:1620] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:47.997738 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=e9ddab232c634786a34bc8ea3b449f19] alter complete (version 1)
I20240426 09:49:47.999258 15998 ts_tablet_manager.cc:902] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:47.999823 15998 tablet_copy_client.cc:308] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:48.000280 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 2c41aab70e4a4b4e86cf095575cd8805 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:32841)
I20240426 09:49:48.000376 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 2c41aab70e4a4b4e86cf095575cd8805 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:32841: session id = e3394a4be9ea4def8aa39b333c82c5ff-2c41aab70e4a4b4e86cf095575cd8805
I20240426 09:49:48.001332 9685 tablet_copy_source_session.cc:205] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:48.002245 15998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c41aab70e4a4b4e86cf095575cd8805. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.004586 15998 tablet_copy_client.cc:783] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:48.004731 15998 tablet_copy_client.cc:647] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:48.007545 15998 tablet_copy_client.cc:523] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:48.010223 15998 tablet_bootstrap.cc:492] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:48.019955 15998 tablet_bootstrap.cc:492] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:48.020212 15998 tablet_bootstrap.cc:492] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:49:48.020377 15998 ts_tablet_manager.cc:1367] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20240426 09:49:48.020831 15998 raft_consensus.cc:348] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.020936 15998 raft_consensus.cc:729] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.021042 15998 consensus_queue.cc:260] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.021319 15998 ts_tablet_manager.cc:1398] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:48.021757 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-2c41aab70e4a4b4e86cf095575cd8805 received from {username='slave'} at 127.0.99.132:32841
I20240426 09:49:48.021855 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-2c41aab70e4a4b4e86cf095575cd8805 on tablet 2c41aab70e4a4b4e86cf095575cd8805 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:48.032932 16024 ts_tablet_manager.cc:902] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:48.033480 16024 tablet_copy_client.cc:308] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:48.033911 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet cd377032e7cd4b4998424f8685ad5b85 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:32985)
I20240426 09:49:48.034011 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet cd377032e7cd4b4998424f8685ad5b85 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:32985: session id = 558c4ecd0ece460d9591aa7a3c799d19-cd377032e7cd4b4998424f8685ad5b85
I20240426 09:49:48.035097 7164 tablet_copy_source_session.cc:205] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:48.037526 16024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd377032e7cd4b4998424f8685ad5b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.042320 16024 tablet_copy_client.cc:783] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:48.042542 16024 tablet_copy_client.cc:647] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:48.044343 16024 tablet_copy_client.cc:523] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:48.045524 16024 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:48.077698 16041 ts_tablet_manager.cc:902] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:48.078231 16041 tablet_copy_client.cc:308] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:48.078692 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 1fa5a98645834e2886ff45b8b28c1c91 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:55679)
I20240426 09:49:48.078826 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 1fa5a98645834e2886ff45b8b28c1c91 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:55679: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-1fa5a98645834e2886ff45b8b28c1c91
I20240426 09:49:48.079741 6902 tablet_copy_source_session.cc:205] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:48.081838 16041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa5a98645834e2886ff45b8b28c1c91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.082098 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:48.083124 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:48.087878 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:48.088106 9529 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:48.088454 9395 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:48.092315 16041 tablet_copy_client.cc:783] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:48.092530 16041 tablet_copy_client.cc:647] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:48.103698 9661 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:48.104646 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:48.105901 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:48.113212 15996 tablet.cc:1722] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:48.113561 15996 tablet.cc:1620] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:48.114857 15994 tablet.cc:1722] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:48.115147 15994 tablet.cc:1620] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:48.117926 15995 tablet.cc:1722] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:48.118207 15995 tablet.cc:1620] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:48.118723 16041 tablet_copy_client.cc:523] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:48.118949 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=e9ddab232c634786a34bc8ea3b449f19] alter complete (version 2)
I20240426 09:49:48.121770 16041 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:48.136759 16024 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:48.137089 16024 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:48.137271 16024 ts_tablet_manager.cc:1367] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.092s user 0.025s sys 0.006s
I20240426 09:49:48.137696 16024 raft_consensus.cc:348] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:48.137815 16024 raft_consensus.cc:729] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.137943 16024 consensus_queue.cc:260] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:48.138242 16024 ts_tablet_manager.cc:1398] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:48.142459 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-cd377032e7cd4b4998424f8685ad5b85 received from {username='slave'} at 127.0.99.136:32985
I20240426 09:49:48.142547 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-cd377032e7cd4b4998424f8685ad5b85 on tablet cd377032e7cd4b4998424f8685ad5b85 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:48.152504 16046 ts_tablet_manager.cc:902] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:48.156693 16046 tablet_copy_client.cc:308] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:48.157130 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 5fe43b0765d9401a88c8e9a028ab3ef4 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:56221)
I20240426 09:49:48.157235 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 5fe43b0765d9401a88c8e9a028ab3ef4 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:56221: session id = 558c4ecd0ece460d9591aa7a3c799d19-5fe43b0765d9401a88c8e9a028ab3ef4
I20240426 09:49:48.158079 6902 tablet_copy_source_session.cc:205] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:48.158983 16046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fe43b0765d9401a88c8e9a028ab3ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.161270 16046 tablet_copy_client.cc:783] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:48.161415 16046 tablet_copy_client.cc:647] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:48.164901 16046 tablet_copy_client.cc:523] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:48.165552 7253 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:48.166792 16046 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:48.177011 16041 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:48.177345 16041 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:48.177603 16041 ts_tablet_manager.cc:1367] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.056s user 0.027s sys 0.000s
I20240426 09:49:48.178129 16041 raft_consensus.cc:348] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:48.178351 16041 raft_consensus.cc:729] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.178560 16041 consensus_queue.cc:260] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:48.178937 16041 ts_tablet_manager.cc:1398] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:48.179893 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-1fa5a98645834e2886ff45b8b28c1c91 received from {username='slave'} at 127.0.99.135:55679
I20240426 09:49:48.179975 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-1fa5a98645834e2886ff45b8b28c1c91 on tablet 1fa5a98645834e2886ff45b8b28c1c91 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:48.192692 16046 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:48.193027 16046 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:48.193248 16046 ts_tablet_manager.cc:1367] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.027s user 0.019s sys 0.003s
I20240426 09:49:48.193961 16046 raft_consensus.cc:348] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:48.194106 16046 raft_consensus.cc:729] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.194252 16046 consensus_queue.cc:260] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:48.194582 16046 ts_tablet_manager.cc:1398] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:48.195412 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-5fe43b0765d9401a88c8e9a028ab3ef4 received from {username='slave'} at 127.0.99.136:56221
I20240426 09:49:48.195488 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-5fe43b0765d9401a88c8e9a028ab3ef4 on tablet 5fe43b0765d9401a88c8e9a028ab3ef4 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:48.206070 15691 consensus_queue.cc:1035] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:48.210556 15950 consensus_queue.cc:1035] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:48.227176 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:48.233469 15994 tablet.cc:1722] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:48.234140 15995 tablet.cc:1722] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:48.236474 15996 tablet.cc:1722] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:48.241612 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=e9ddab232c634786a34bc8ea3b449f19] alter complete (version 3)
I20240426 09:49:48.250686 15494 consensus_queue.cc:1035] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:48.252593 15494 consensus_queue.cc:1035] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:48.260341 15952 consensus_queue.cc:1035] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:48.268149 15952 consensus_queue.cc:1035] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:48.273705 6733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:48.287619 6860 tablet_service.cc:1467] Processing CreateTablet for tablet b967d83e93d843398e93077285865b1b (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:48.287935 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b967d83e93d843398e93077285865b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.290364 16105 tablet_bootstrap.cc:492] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:48.291663 16105 tablet_bootstrap.cc:654] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.300791 9395 tablet_service.cc:1467] Processing CreateTablet for tablet 6404a6d30b534fe3a1748a2e486579fc (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:48.301124 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6404a6d30b534fe3a1748a2e486579fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.301352 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 6404a6d30b534fe3a1748a2e486579fc (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:48.301616 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6404a6d30b534fe3a1748a2e486579fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.303054 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 6404a6d30b534fe3a1748a2e486579fc (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:48.303344 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6404a6d30b534fe3a1748a2e486579fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.305235 16105 tablet_bootstrap.cc:492] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:48.305377 16105 ts_tablet_manager.cc:1367] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240426 09:49:48.305716 7255 tablet_service.cc:1467] Processing CreateTablet for tablet d83c01e6dc6d4b5b9365243058f459a1 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:48.305845 6991 tablet_service.cc:1467] Processing CreateTablet for tablet b967d83e93d843398e93077285865b1b (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:48.306003 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83c01e6dc6d4b5b9365243058f459a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.305939 16105 raft_consensus.cc:348] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.306051 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b967d83e93d843398e93077285865b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.306049 16105 raft_consensus.cc:374] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.306077 16105 raft_consensus.cc:729] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.306210 16105 consensus_queue.cc:260] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.306468 16105 ts_tablet_manager.cc:1398] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:48.308066 7122 tablet_service.cc:1467] Processing CreateTablet for tablet d83c01e6dc6d4b5b9365243058f459a1 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:48.308311 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83c01e6dc6d4b5b9365243058f459a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.313056 9266 tablet_service.cc:1467] Processing CreateTablet for tablet d83c01e6dc6d4b5b9365243058f459a1 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:48.313310 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83c01e6dc6d4b5b9365243058f459a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.314499 7253 tablet_service.cc:1467] Processing CreateTablet for tablet b967d83e93d843398e93077285865b1b (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:48.314810 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b967d83e93d843398e93077285865b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.319998 16107 tablet_bootstrap.cc:492] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:48.321230 16113 tablet_bootstrap.cc:492] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:48.322243 16113 tablet_bootstrap.cc:654] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.323613 16113 tablet_bootstrap.cc:492] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:48.323704 16113 ts_tablet_manager.cc:1367] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:48.324196 16113 raft_consensus.cc:348] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.324312 16113 raft_consensus.cc:374] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.324354 16113 raft_consensus.cc:729] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.324492 16113 consensus_queue.cc:260] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.324771 16113 ts_tablet_manager.cc:1398] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:48.324934 16113 tablet_bootstrap.cc:492] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:48.325968 16113 tablet_bootstrap.cc:654] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.329731 16108 tablet_bootstrap.cc:492] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:48.330634 16108 tablet_bootstrap.cc:654] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.332165 16109 tablet_bootstrap.cc:492] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:48.332382 16113 tablet_bootstrap.cc:492] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:48.332473 16113 ts_tablet_manager.cc:1367] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240426 09:49:48.332926 16113 raft_consensus.cc:348] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.333024 16113 raft_consensus.cc:374] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.333045 16113 raft_consensus.cc:729] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.333106 16109 tablet_bootstrap.cc:654] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.333156 16113 consensus_queue.cc:260] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.333434 16113 ts_tablet_manager.cc:1398] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:48.321264 16110 tablet_bootstrap.cc:492] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:48.338629 16110 tablet_bootstrap.cc:654] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.339294 16107 tablet_bootstrap.cc:654] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.339812 16109 tablet_bootstrap.cc:492] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:48.340840 16109 ts_tablet_manager.cc:1367] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:48.341486 16110 tablet_bootstrap.cc:492] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:48.341617 16107 tablet_bootstrap.cc:492] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:48.341626 16110 ts_tablet_manager.cc:1367] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240426 09:49:48.342278 16110 raft_consensus.cc:348] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:48.343214 16110 raft_consensus.cc:374] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.343199 6735 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:48.343304 16110 raft_consensus.cc:729] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.343276 16109 raft_consensus.cc:348] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.343354 6735 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:48.343379 16109 raft_consensus.cc:374] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.343406 16109 raft_consensus.cc:729] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.343521 16109 consensus_queue.cc:260] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.343480 16110 consensus_queue.cc:260] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:48.344761 16108 tablet_bootstrap.cc:492] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:48.344839 16108 ts_tablet_manager.cc:1367] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20240426 09:49:48.344862 16110 ts_tablet_manager.cc:1398] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:48.345036 16110 tablet_bootstrap.cc:492] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:48.345423 16108 raft_consensus.cc:348] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:48.345544 16108 raft_consensus.cc:374] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.345575 16108 raft_consensus.cc:729] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.345727 16108 consensus_queue.cc:260] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:48.346030 16108 ts_tablet_manager.cc:1398] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:48.346223 16109 ts_tablet_manager.cc:1398] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:48.347710 16107 ts_tablet_manager.cc:1367] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.000s
I20240426 09:49:48.348331 16110 tablet_bootstrap.cc:654] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.348780 16107 raft_consensus.cc:348] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:48.348960 16107 raft_consensus.cc:374] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.348999 16107 raft_consensus.cc:729] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.349131 16107 consensus_queue.cc:260] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:48.349576 16107 ts_tablet_manager.cc:1398] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240426 09:49:48.349771 6735 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 82c3f4fc1e304580a287c0fc031614db
I20240426 09:49:48.350536 9529 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c3f4fc1e304580a287c0fc031614db with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:48 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:48.350606 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c3f4fc1e304580a287c0fc031614db with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:48 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:48.350816 6860 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c3f4fc1e304580a287c0fc031614db with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:48 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:48.355201 16130 tablet_replica.cc:330] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:48.355361 16130 raft_consensus.cc:2229] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:48.355512 16130 raft_consensus.cc:2258] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:48.355779 16128 tablet_replica.cc:330] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:48.356215 16130 ts_tablet_manager.cc:1875] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:48.356534 16128 raft_consensus.cc:2229] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:48.357630 16128 raft_consensus.cc:2258] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:48.362217 16128 ts_tablet_manager.cc:1875] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:48.367164 16130 ts_tablet_manager.cc:1888] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:48.367672 16130 log.cc:1196] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/82c3f4fc1e304580a287c0fc031614db
I20240426 09:49:48.368043 16128 ts_tablet_manager.cc:1888] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:48.369940 16128 log.cc:1196] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/82c3f4fc1e304580a287c0fc031614db
I20240426 09:49:48.371057 16128 ts_tablet_manager.cc:1909] T 82c3f4fc1e304580a287c0fc031614db P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:48.371516 16130 ts_tablet_manager.cc:1909] T 82c3f4fc1e304580a287c0fc031614db P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:48.372820 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 82c3f4fc1e304580a287c0fc031614db (table rebalancer_dynamic_table_new_name [id=e9ddab232c634786a34bc8ea3b449f19]) successfully deleted
I20240426 09:49:48.372927 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 82c3f4fc1e304580a287c0fc031614db (table rebalancer_dynamic_table_new_name [id=e9ddab232c634786a34bc8ea3b449f19]) successfully deleted
I20240426 09:49:48.377312 6860 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:48.377946 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:48.397956 9529 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:48.398239 7255 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:48.401719 9661 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
W20240426 09:49:48.404095 7191 tablet.cc:2369] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:48.406688 16108 tablet_bootstrap.cc:492] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:48.409013 16108 tablet_bootstrap.cc:654] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.409206 16129 tablet_replica.cc:330] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:48.409940 16129 raft_consensus.cc:2229] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:48.410177 16129 raft_consensus.cc:2258] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:48.411618 16110 tablet_bootstrap.cc:492] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:48.411757 16110 ts_tablet_manager.cc:1367] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.067s user 0.002s sys 0.000s
I20240426 09:49:48.412281 16110 raft_consensus.cc:348] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.413597 16129 ts_tablet_manager.cc:1875] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:48.412461 16110 raft_consensus.cc:374] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.413790 16110 raft_consensus.cc:729] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.414551 16110 consensus_queue.cc:260] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.416186 6991 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:48.417200 9395 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240426 09:49:48.418665 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:48.420117 16110 ts_tablet_manager.cc:1398] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:49:48.421402 16129 ts_tablet_manager.cc:1888] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:48.423599 16129 log.cc:1196] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/82c3f4fc1e304580a287c0fc031614db
I20240426 09:49:48.432525 16108 tablet_bootstrap.cc:492] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:48.434625 16108 ts_tablet_manager.cc:1367] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.002s
I20240426 09:49:48.435129 16108 raft_consensus.cc:348] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.435236 16108 raft_consensus.cc:374] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.435264 16108 raft_consensus.cc:729] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.435369 16108 consensus_queue.cc:260] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.435619 16108 ts_tablet_manager.cc:1398] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:48.437117 16129 ts_tablet_manager.cc:1909] T 82c3f4fc1e304580a287c0fc031614db P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:48.445230 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 82c3f4fc1e304580a287c0fc031614db (table rebalancer_dynamic_table_new_name [id=e9ddab232c634786a34bc8ea3b449f19]) successfully deleted
I20240426 09:49:48.460083 6736 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:48.467808 6991 tablet_service.cc:1467] Processing CreateTablet for tablet c960a49f5f70486c95d568813139a019 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6b6c13ed34a4460bae7b73771d6242a0]), partition=
I20240426 09:49:48.468092 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c960a49f5f70486c95d568813139a019. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.468253 7122 tablet_service.cc:1467] Processing CreateTablet for tablet c960a49f5f70486c95d568813139a019 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6b6c13ed34a4460bae7b73771d6242a0]), partition=
I20240426 09:49:48.468506 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c960a49f5f70486c95d568813139a019. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.469990 16108 tablet_bootstrap.cc:492] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:48.470299 9266 tablet_service.cc:1467] Processing CreateTablet for tablet c960a49f5f70486c95d568813139a019 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6b6c13ed34a4460bae7b73771d6242a0]), partition=
I20240426 09:49:48.470522 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c960a49f5f70486c95d568813139a019. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:48.470896 16108 tablet_bootstrap.cc:654] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.472615 16108 tablet_bootstrap.cc:492] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:48.472687 16108 ts_tablet_manager.cc:1367] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:48.472819 16110 tablet_bootstrap.cc:492] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:48.472908 16109 tablet_bootstrap.cc:492] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:48.473145 16108 raft_consensus.cc:348] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.473245 16108 raft_consensus.cc:374] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.473284 16108 raft_consensus.cc:729] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.473439 16108 consensus_queue.cc:260] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.473774 16110 tablet_bootstrap.cc:654] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.473953 16109 tablet_bootstrap.cc:654] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:48.474951 16108 ts_tablet_manager.cc:1398] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:48.475917 16110 tablet_bootstrap.cc:492] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:48.475931 16109 tablet_bootstrap.cc:492] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:48.475998 16110 ts_tablet_manager.cc:1367] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:48.476013 16109 ts_tablet_manager.cc:1367] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:48.476346 16110 raft_consensus.cc:348] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.476349 16109 raft_consensus.cc:348] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.476421 16110 raft_consensus.cc:374] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.476428 16109 raft_consensus.cc:374] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:48.476442 16110 raft_consensus.cc:729] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.476449 16109 raft_consensus.cc:729] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:48.476550 16109 consensus_queue.cc:260] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.476541 16110 consensus_queue.cc:260] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.476832 16110 ts_tablet_manager.cc:1398] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:48.476831 16109 ts_tablet_manager.cc:1398] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20240426 09:49:48.478355 7060 tablet.cc:2369] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:48.510888 15391 raft_consensus.cc:482] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:48.511016 15391 raft_consensus.cc:504] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.511456 15391 leader_election.cc:288] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:48.512019 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83c01e6dc6d4b5b9365243058f459a1" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:48.512168 7143 raft_consensus.cc:2454] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:48.512460 7208 leader_election.cc:302] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:48.512460 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83c01e6dc6d4b5b9365243058f459a1" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:48.512588 9286 raft_consensus.cc:2454] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:48.512627 15391 raft_consensus.cc:2781] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:48.512712 15391 raft_consensus.cc:482] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:48.512745 15391 raft_consensus.cc:3037] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.513828 15391 raft_consensus.cc:504] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.514161 15391 leader_election.cc:288] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:48.514513 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83c01e6dc6d4b5b9365243058f459a1" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:48.514626 9286 raft_consensus.cc:3037] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.515657 9286 raft_consensus.cc:2454] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:48.515960 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83c01e6dc6d4b5b9365243058f459a1" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:48.516062 7143 raft_consensus.cc:3037] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.517067 7143 raft_consensus.cc:2454] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:48.517588 7209 leader_election.cc:302] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:48.517737 15391 raft_consensus.cc:2781] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:48.518242 15391 raft_consensus.cc:686] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:48.518374 15391 consensus_queue.cc:237] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.520821 6735 catalog_manager.cc:5484] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:48.530632 15595 raft_consensus.cc:482] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:48.530746 15595 raft_consensus.cc:504] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.531267 15595 leader_election.cc:288] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:48.532042 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b967d83e93d843398e93077285865b1b" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:48.532042 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b967d83e93d843398e93077285865b1b" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:48.532167 7011 raft_consensus.cc:2454] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:48.532167 7274 raft_consensus.cc:2454] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:48.532459 6817 leader_election.cc:302] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:48.532678 15595 raft_consensus.cc:2781] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:48.532739 15595 raft_consensus.cc:482] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:48.532784 15595 raft_consensus.cc:3037] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.533878 15595 raft_consensus.cc:504] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.534235 15595 leader_election.cc:288] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:48.534500 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b967d83e93d843398e93077285865b1b" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:48.534597 7011 raft_consensus.cc:3037] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.534637 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b967d83e93d843398e93077285865b1b" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:48.534747 7274 raft_consensus.cc:3037] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.535681 7011 raft_consensus.cc:2454] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:48.536561 7274 raft_consensus.cc:2454] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:48.536849 6817 leader_election.cc:302] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:48.537074 15595 raft_consensus.cc:2781] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:48.537165 15595 raft_consensus.cc:686] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:48.537619 15595 consensus_queue.cc:237] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:48.541450 6734 catalog_manager.cc:5484] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
W20240426 09:49:48.550220 6929 tablet.cc:2369] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:48.622933 7274 raft_consensus.cc:1203] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:48.637960 9672 raft_consensus.cc:1203] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:48.671627 15691 raft_consensus.cc:482] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:48.671747 15691 raft_consensus.cc:504] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:48.671991 15865 raft_consensus.cc:482] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:48.672096 15865 raft_consensus.cc:504] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.672200 15691 leader_election.cc:288] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:48.672498 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6404a6d30b534fe3a1748a2e486579fc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:48.672606 15865 leader_election.cc:288] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:48.672641 7011 raft_consensus.cc:2454] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:48.672618 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6404a6d30b534fe3a1748a2e486579fc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:48.672744 9286 raft_consensus.cc:2454] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:48.672961 9350 leader_election.cc:302] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:48.673013 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c960a49f5f70486c95d568813139a019" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:48.673139 7011 raft_consensus.cc:2454] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:48.673182 15691 raft_consensus.cc:2781] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:48.673180 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c960a49f5f70486c95d568813139a019" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:48.673230 15691 raft_consensus.cc:482] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:48.673254 15691 raft_consensus.cc:3037] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.673272 9286 raft_consensus.cc:2454] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:48.673389 7077 leader_election.cc:302] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:48.673624 15865 raft_consensus.cc:2781] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:48.673679 15865 raft_consensus.cc:482] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:48.673733 15865 raft_consensus.cc:3037] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.674285 15691 raft_consensus.cc:504] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:48.674471 9549 raft_consensus.cc:1203] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:48.674623 15691 leader_election.cc:288] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:48.674650 15865 raft_consensus.cc:504] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.675371 15865 leader_election.cc:288] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:48.675776 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c960a49f5f70486c95d568813139a019" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:48.675807 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6404a6d30b534fe3a1748a2e486579fc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:48.675880 9285 raft_consensus.cc:3037] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.675913 9286 raft_consensus.cc:3037] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.676604 9286 raft_consensus.cc:2454] T 6404a6d30b534fe3a1748a2e486579fc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:48.676805 9285 raft_consensus.cc:2454] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:48.676803 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6404a6d30b534fe3a1748a2e486579fc" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:48.676898 7013 raft_consensus.cc:3037] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.677371 7078 leader_election.cc:302] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:48.677418 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c960a49f5f70486c95d568813139a019" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:48.677515 7011 raft_consensus.cc:3037] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:48.677529 15865 raft_consensus.cc:2781] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:48.677716 15865 raft_consensus.cc:686] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:48.677886 7013 raft_consensus.cc:2454] T 6404a6d30b534fe3a1748a2e486579fc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:48.677860 15865 consensus_queue.cc:237] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:48.678169 9351 leader_election.cc:302] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:48.678326 15691 raft_consensus.cc:2781] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:48.678395 15691 raft_consensus.cc:686] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:48.678468 7011 raft_consensus.cc:2454] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:48.678506 15691 consensus_queue.cc:237] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:48.680645 6735 catalog_manager.cc:5484] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:48.681771 6736 catalog_manager.cc:5484] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:48.685588 9672 raft_consensus.cc:1203] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:48.702649 6991 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:48.703989 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:48.707906 9661 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:48.710250 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:48.714236 6860 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:48.730029 9529 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:48.733232 7255 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:48.735738 9395 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:48.800467 7274 consensus_queue.cc:237] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:48.801714 7011 raft_consensus.cc:1261] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:48.801805 9672 raft_consensus.cc:1261] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:48.802090 15706 consensus_queue.cc:1035] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:48.802639 15706 consensus_queue.cc:1035] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:48.804235 15391 raft_consensus.cc:2932] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:48.804731 7011 raft_consensus.cc:2932] T 325e1d125617456f877204314d9e7bb9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:48.804731 9672 raft_consensus.cc:2932] T 325e1d125617456f877204314d9e7bb9 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:48.807296 6736 catalog_manager.cc:5484] T 325e1d125617456f877204314d9e7bb9 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:48.810101 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 325e1d125617456f877204314d9e7bb9 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:48.810542 16219 tablet_replica.cc:330] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:48.810693 16219 raft_consensus.cc:2229] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:48.810889 16219 raft_consensus.cc:2258] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:48.811491 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:48.811836 16219 ts_tablet_manager.cc:1875] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:48.816164 9285 raft_consensus.cc:1261] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:48.816530 7011 raft_consensus.cc:1261] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:48.816613 15865 consensus_queue.cc:1035] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:48.816879 15865 consensus_queue.cc:1035] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:48.819732 16219 ts_tablet_manager.cc:1888] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:48.819885 16219 log.cc:1196] T 325e1d125617456f877204314d9e7bb9 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/325e1d125617456f877204314d9e7bb9
I20240426 09:49:48.820000 6739 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet c960a49f5f70486c95d568813139a019 (table rebalancer_dynamic_table [id=6b6c13ed34a4460bae7b73771d6242a0]). Expected version 1 got 0
I20240426 09:49:48.820444 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 325e1d125617456f877204314d9e7bb9 (table rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]) successfully deleted
I20240426 09:49:48.820914 16233 tablet.cc:1722] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:48.821120 6735 catalog_manager.cc:5576] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135) does not have the latest schema for tablet c960a49f5f70486c95d568813139a019 (table rebalancer_dynamic_table [id=6b6c13ed34a4460bae7b73771d6242a0]). Expected version 1 got 0
I20240426 09:49:48.821465 16233 tablet.cc:1620] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:48.822986 16234 tablet.cc:1722] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:48.823402 16234 tablet.cc:1620] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:48.824368 16232 tablet.cc:1722] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:48.824656 16232 tablet.cc:1620] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:48.826961 6735 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet c960a49f5f70486c95d568813139a019 (table rebalancer_dynamic_table [id=6b6c13ed34a4460bae7b73771d6242a0]). Expected version 1 got 0
I20240426 09:49:48.827486 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=6b6c13ed34a4460bae7b73771d6242a0] alter complete (version 1)
I20240426 09:49:48.930122 9661 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:48.930217 6733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:48.930370 9395 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:48.931294 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:48.934933 6991 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:48.936244 16232 tablet.cc:1722] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:48.936578 16232 tablet.cc:1620] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:48.936671 16233 tablet.cc:1722] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:48.936947 16233 tablet.cc:1620] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:48.937279 16234 tablet.cc:1722] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:48.937534 16234 tablet.cc:1620] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:48.942894 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=6b6c13ed34a4460bae7b73771d6242a0] alter complete (version 2)
I20240426 09:49:48.944523 6860 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:48.958299 7255 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:48.962270 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:48.983508 15595 consensus_queue.cc:1035] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:48.990968 9529 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:49.006695 15391 consensus_queue.cc:1035] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:49.013684 15391 consensus_queue.cc:1035] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:49.023144 15178 consensus_queue.cc:1035] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:49.045651 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:49.051470 16232 tablet.cc:1722] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:49.051957 16233 tablet.cc:1722] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:49.052546 16234 tablet.cc:1722] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:49.058827 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=6b6c13ed34a4460bae7b73771d6242a0] alter complete (version 3)
I20240426 09:49:49.060773 7011 consensus_queue.cc:237] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.062304 9672 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:49:49.062343 7142 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:49:49.062979 15237 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:49:49.063511 15237 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:49:49.066365 15670 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:49.066378 9672 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:49.068750 7142 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:49.072065 6733 catalog_manager.cc:5484] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 4 to 6, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:49.079981 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 643fd2d359a14f8eaa8d8a0d0879ef61 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:49.080412 16309 tablet_replica.cc:330] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:49.080615 16309 raft_consensus.cc:2229] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:49.080749 16309 raft_consensus.cc:2258] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:49.081358 16309 ts_tablet_manager.cc:1875] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:49.084372 16309 ts_tablet_manager.cc:1888] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:49:49.084517 16309 log.cc:1196] T 643fd2d359a14f8eaa8d8a0d0879ef61 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/643fd2d359a14f8eaa8d8a0d0879ef61
I20240426 09:49:49.085168 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 643fd2d359a14f8eaa8d8a0d0879ef61 (table rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]) successfully deleted
I20240426 09:49:49.095625 15950 consensus_queue.cc:1035] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:49.107060 15950 consensus_queue.cc:1035] T 6404a6d30b534fe3a1748a2e486579fc P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:49.160271 6732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:49.160425 6732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:49.164418 6732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet c960a49f5f70486c95d568813139a019
I20240426 09:49:49.165021 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet c960a49f5f70486c95d568813139a019 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:49 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:49.165417 16349 tablet_replica.cc:330] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:49.165611 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet c960a49f5f70486c95d568813139a019 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:49 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:49.166040 16349 raft_consensus.cc:2229] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:49.166298 16349 raft_consensus.cc:2258] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:49.166749 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet c960a49f5f70486c95d568813139a019 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:49 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:49.167153 16219 tablet_replica.cc:330] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:49.167200 16350 tablet_replica.cc:330] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:49.167310 16219 raft_consensus.cc:2229] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:49.167320 16350 raft_consensus.cc:2229] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:49.167446 16350 raft_consensus.cc:2258] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:49.167526 16219 raft_consensus.cc:2258] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:49.168377 16350 ts_tablet_manager.cc:1875] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:49.170045 16219 ts_tablet_manager.cc:1875] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:49.172418 16219 ts_tablet_manager.cc:1888] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:49.172484 16219 log.cc:1196] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/c960a49f5f70486c95d568813139a019
I20240426 09:49:49.172756 16219 ts_tablet_manager.cc:1909] T c960a49f5f70486c95d568813139a019 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:49.178560 16349 ts_tablet_manager.cc:1875] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:49.179136 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet c960a49f5f70486c95d568813139a019 (table rebalancer_dynamic_table_new_name [id=6b6c13ed34a4460bae7b73771d6242a0]) successfully deleted
I20240426 09:49:49.181210 16349 ts_tablet_manager.cc:1888] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:49.181344 16349 log.cc:1196] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/c960a49f5f70486c95d568813139a019
I20240426 09:49:49.181681 16349 ts_tablet_manager.cc:1909] T c960a49f5f70486c95d568813139a019 P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:49.182407 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet c960a49f5f70486c95d568813139a019 (table rebalancer_dynamic_table_new_name [id=6b6c13ed34a4460bae7b73771d6242a0]) successfully deleted
I20240426 09:49:49.182603 16350 ts_tablet_manager.cc:1888] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:49.182680 16350 log.cc:1196] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/c960a49f5f70486c95d568813139a019
I20240426 09:49:49.183019 16350 ts_tablet_manager.cc:1909] T c960a49f5f70486c95d568813139a019 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:49.184100 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet c960a49f5f70486c95d568813139a019 (table rebalancer_dynamic_table_new_name [id=6b6c13ed34a4460bae7b73771d6242a0]) successfully deleted
I20240426 09:49:49.204422 6991 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:49.210898 9661 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:49.211688 7122 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:49.212308 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:49.214438 9395 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:49.231449 6860 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:49.232729 6731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:49.239977 7255 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:49.246704 9661 tablet_service.cc:1467] Processing CreateTablet for tablet b925f8cd837b4e5f8fd3f6f608d3ad31 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:49.247045 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b925f8cd837b4e5f8fd3f6f608d3ad31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.247236 9529 tablet_service.cc:1467] Processing CreateTablet for tablet b925f8cd837b4e5f8fd3f6f608d3ad31 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:49.247501 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b925f8cd837b4e5f8fd3f6f608d3ad31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.249461 16362 tablet_bootstrap.cc:492] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:49.249833 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 635293adcd7b45858935a65e4ba58f24 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:49.250075 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635293adcd7b45858935a65e4ba58f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.250339 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 635293adcd7b45858935a65e4ba58f24 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:49.250491 6991 tablet_service.cc:1467] Processing CreateTablet for tablet a31b0f75c2514916a94205ce0289629e (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:49.250630 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635293adcd7b45858935a65e4ba58f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.250818 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a31b0f75c2514916a94205ce0289629e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.252348 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 635293adcd7b45858935a65e4ba58f24 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:49.252614 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 635293adcd7b45858935a65e4ba58f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.253135 9266 tablet_service.cc:1467] Processing CreateTablet for tablet b925f8cd837b4e5f8fd3f6f608d3ad31 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:49.253347 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b925f8cd837b4e5f8fd3f6f608d3ad31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.254118 16362 tablet_bootstrap.cc:654] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.255684 6860 tablet_service.cc:1467] Processing CreateTablet for tablet a31b0f75c2514916a94205ce0289629e (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:49.255971 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a31b0f75c2514916a94205ce0289629e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.256239 9527 tablet_service.cc:1467] Processing CreateTablet for tablet a31b0f75c2514916a94205ce0289629e (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:49.256481 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a31b0f75c2514916a94205ce0289629e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.265923 6731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:49.268306 16368 tablet_bootstrap.cc:492] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:49.269510 16368 tablet_bootstrap.cc:654] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.270006 9527 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:49.270722 16368 tablet_bootstrap.cc:492] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:49.270860 16368 ts_tablet_manager.cc:1367] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:49.271323 16368 raft_consensus.cc:348] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:49.271431 16368 raft_consensus.cc:374] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.271474 16368 raft_consensus.cc:729] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.271603 16368 consensus_queue.cc:260] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:49.271910 16368 ts_tablet_manager.cc:1398] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:49.276688 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 07a10ef63f1f442d974c27d5846dd638 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2d023942a6174555a6b2a1633ff0e709]), partition=
I20240426 09:49:49.276923 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a10ef63f1f442d974c27d5846dd638. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.282963 16365 tablet_bootstrap.cc:492] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:49.284037 16365 tablet_bootstrap.cc:654] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.284348 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 07a10ef63f1f442d974c27d5846dd638 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2d023942a6174555a6b2a1633ff0e709]), partition=
I20240426 09:49:49.284487 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 07a10ef63f1f442d974c27d5846dd638 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2d023942a6174555a6b2a1633ff0e709]), partition=
I20240426 09:49:49.284576 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a10ef63f1f442d974c27d5846dd638. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.284749 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a10ef63f1f442d974c27d5846dd638. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.284817 16363 tablet_bootstrap.cc:492] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:49.285748 16364 tablet_bootstrap.cc:492] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:49.286799 16363 tablet_bootstrap.cc:654] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.291329 16364 tablet_bootstrap.cc:654] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.292327 16363 tablet_bootstrap.cc:492] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:49.292423 16363 ts_tablet_manager.cc:1367] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240426 09:49:49.292811 16363 raft_consensus.cc:348] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.292925 16363 raft_consensus.cc:374] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.292959 16363 raft_consensus.cc:729] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.293087 16363 consensus_queue.cc:260] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.293601 16364 tablet_bootstrap.cc:492] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:49.293689 16364 ts_tablet_manager.cc:1367] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:49.293712 16365 tablet_bootstrap.cc:492] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:49.294113 16365 ts_tablet_manager.cc:1367] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:49.294098 16364 raft_consensus.cc:348] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:49.294207 16364 raft_consensus.cc:374] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.294250 16364 raft_consensus.cc:729] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.294377 16364 consensus_queue.cc:260] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:49.294996 16363 ts_tablet_manager.cc:1398] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:49:49.294692 16365 raft_consensus.cc:348] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:49.295202 16363 tablet_bootstrap.cc:492] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:49.295243 16370 tablet_bootstrap.cc:492] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:49.295267 16365 raft_consensus.cc:374] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.295338 16365 raft_consensus.cc:729] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.295483 16365 consensus_queue.cc:260] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:49.295866 16365 ts_tablet_manager.cc:1398] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:49.296294 16363 tablet_bootstrap.cc:654] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.296551 16366 tablet_bootstrap.cc:492] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:49.294713 16364 ts_tablet_manager.cc:1398] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:49.296806 16364 tablet_bootstrap.cc:492] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:49.297423 16370 tablet_bootstrap.cc:654] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.297823 16366 tablet_bootstrap.cc:654] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.301007 16364 tablet_bootstrap.cc:654] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.304558 16366 tablet_bootstrap.cc:492] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:49.304590 16362 tablet_bootstrap.cc:492] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:49.304692 16366 ts_tablet_manager.cc:1367] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:49.304692 16362 ts_tablet_manager.cc:1367] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.000s
I20240426 09:49:49.305172 16366 raft_consensus.cc:348] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:49.305276 16366 raft_consensus.cc:374] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.305313 16366 raft_consensus.cc:729] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.305430 16366 consensus_queue.cc:260] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:49.305773 16366 ts_tablet_manager.cc:1398] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:49.306073 16366 tablet_bootstrap.cc:492] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:49.306964 16370 tablet_bootstrap.cc:492] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:49.306988 16366 tablet_bootstrap.cc:654] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.307051 16370 ts_tablet_manager.cc:1367] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:49:49.307677 16363 tablet_bootstrap.cc:492] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:49.307798 16363 ts_tablet_manager.cc:1367] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20240426 09:49:49.307754 16370 raft_consensus.cc:348] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.307876 16370 raft_consensus.cc:374] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.307919 16370 raft_consensus.cc:729] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.308283 16370 consensus_queue.cc:260] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.305151 16362 raft_consensus.cc:348] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.308645 16362 raft_consensus.cc:374] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.308697 16362 raft_consensus.cc:729] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.308821 16362 consensus_queue.cc:260] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.309147 16362 ts_tablet_manager.cc:1398] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:49.309340 16362 tablet_bootstrap.cc:492] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:49.308676 16363 raft_consensus.cc:348] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:49.310292 16362 tablet_bootstrap.cc:654] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.310320 16363 raft_consensus.cc:374] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.310377 16363 raft_consensus.cc:729] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.310477 16363 consensus_queue.cc:260] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:49.310734 16363 ts_tablet_manager.cc:1398] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20240426 09:49:49.310899 16363 tablet_bootstrap.cc:492] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:49.312485 16363 tablet_bootstrap.cc:654] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:49.313558 16362 tablet_bootstrap.cc:492] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:49.313689 16362 ts_tablet_manager.cc:1367] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:49.314162 16362 raft_consensus.cc:348] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.314318 16362 raft_consensus.cc:374] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.314409 16362 raft_consensus.cc:729] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.314626 16370 ts_tablet_manager.cc:1398] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20240426 09:49:49.314594 16362 consensus_queue.cc:260] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.314905 16362 ts_tablet_manager.cc:1398] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:49.316205 16363 tablet_bootstrap.cc:492] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:49.316335 16363 ts_tablet_manager.cc:1367] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:49.317049 16363 raft_consensus.cc:348] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:49.317200 16363 raft_consensus.cc:374] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.317296 16363 raft_consensus.cc:729] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.317673 16364 tablet_bootstrap.cc:492] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:49.317768 16364 ts_tablet_manager.cc:1367] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240426 09:49:49.317768 16363 consensus_queue.cc:260] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:49.318025 16363 ts_tablet_manager.cc:1398] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:49.318231 16364 raft_consensus.cc:348] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.318339 16364 raft_consensus.cc:374] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.318375 16364 raft_consensus.cc:729] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.318492 16364 consensus_queue.cc:260] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.318740 16364 ts_tablet_manager.cc:1398] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:49.323164 16366 tablet_bootstrap.cc:492] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:49.323333 16366 ts_tablet_manager.cc:1367] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20240426 09:49:49.325372 16366 raft_consensus.cc:348] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.325505 16366 raft_consensus.cc:374] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:49.325551 16366 raft_consensus.cc:729] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:49.325670 16366 consensus_queue.cc:260] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.325940 16366 ts_tablet_manager.cc:1398] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
W20240426 09:49:49.341580 7322 tablet.cc:2369] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:49:49.353041 6882 consensus_queue.cc:237] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:49.354485 9547 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:49.355146 15178 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:49.355449 7142 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:49.355877 15178 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:49.357057 15178 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:49.358992 9547 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:49.359267 7142 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:49.363304 6735 catalog_manager.cc:5484] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:49.366328 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa5a98645834e2886ff45b8b28c1c91 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:49.366549 16350 tablet_replica.cc:330] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:49.366684 16350 raft_consensus.cc:2229] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:49.366863 16350 raft_consensus.cc:2258] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:49.367434 16350 ts_tablet_manager.cc:1875] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:49.369679 16350 ts_tablet_manager.cc:1888] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:49.369755 16350 log.cc:1196] T 1fa5a98645834e2886ff45b8b28c1c91 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/1fa5a98645834e2886ff45b8b28c1c91
I20240426 09:49:49.370332 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 1fa5a98645834e2886ff45b8b28c1c91 (table rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]) successfully deleted
I20240426 09:49:49.402468 7142 consensus_queue.cc:237] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:49.404111 6882 raft_consensus.cc:1261] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:49.404187 7011 raft_consensus.cc:1261] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:49.404444 7077 consensus_peers.cc:485] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 956f89ba28c54473964cc877c26580f0. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:49.404510 16277 consensus_queue.cc:1035] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:49.404675 15865 consensus_queue.cc:1035] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:49.407094 16277 raft_consensus.cc:2932] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:49.407419 6882 raft_consensus.cc:2932] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:49.409626 7013 raft_consensus.cc:2932] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:49.412088 6733 catalog_manager.cc:5484] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:49.482322 16277 raft_consensus.cc:482] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:49.482460 16277 raft_consensus.cc:504] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:49.482905 16277 leader_election.cc:288] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:49.483145 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635293adcd7b45858935a65e4ba58f24" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:49.483284 9547 raft_consensus.cc:2454] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:49.483325 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635293adcd7b45858935a65e4ba58f24" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:49.483476 9285 raft_consensus.cc:2454] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:49.483639 7077 leader_election.cc:302] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:49.483790 16277 raft_consensus.cc:2781] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:49.483840 16277 raft_consensus.cc:482] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:49.483894 16277 raft_consensus.cc:3037] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.484920 16277 raft_consensus.cc:504] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:49.485240 16277 leader_election.cc:288] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:49.485577 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635293adcd7b45858935a65e4ba58f24" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:49.485683 9547 raft_consensus.cc:3037] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.486011 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "635293adcd7b45858935a65e4ba58f24" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:49.486126 9285 raft_consensus.cc:3037] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.486656 9547 raft_consensus.cc:2454] T 635293adcd7b45858935a65e4ba58f24 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:49.486943 7077 leader_election.cc:302] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:49.487088 16277 raft_consensus.cc:2781] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:49.487110 9285 raft_consensus.cc:2454] T 635293adcd7b45858935a65e4ba58f24 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:49.487161 16277 raft_consensus.cc:686] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:49.487293 16277 consensus_queue.cc:237] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:49.490489 6739 catalog_manager.cc:5484] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:49.561028 16112 raft_consensus.cc:482] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:49.561156 16112 raft_consensus.cc:504] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.561674 16112 leader_election.cc:288] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:49.562036 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b925f8cd837b4e5f8fd3f6f608d3ad31" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:49.562175 9669 raft_consensus.cc:2454] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:49.562245 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b925f8cd837b4e5f8fd3f6f608d3ad31" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:49.562377 9285 raft_consensus.cc:2454] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:49.562604 9482 leader_election.cc:302] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:49.562867 16112 raft_consensus.cc:2781] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:49.562922 16112 raft_consensus.cc:482] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:49.562963 16112 raft_consensus.cc:3037] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.563999 16112 raft_consensus.cc:504] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.564339 16112 leader_election.cc:288] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:49.564620 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b925f8cd837b4e5f8fd3f6f608d3ad31" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:49.564667 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b925f8cd837b4e5f8fd3f6f608d3ad31" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:49.564752 9285 raft_consensus.cc:3037] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.564774 9669 raft_consensus.cc:3037] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.565730 9285 raft_consensus.cc:2454] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:49.566015 9483 leader_election.cc:302] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:49.566185 16112 raft_consensus.cc:2781] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:49.566197 9669 raft_consensus.cc:2454] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:49.566265 16112 raft_consensus.cc:686] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:49.566401 16112 consensus_queue.cc:237] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.569500 6738 catalog_manager.cc:5484] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:49.646668 14361 raft_consensus.cc:482] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:49.646817 14361 raft_consensus.cc:504] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.647199 14361 leader_election.cc:288] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:49.647468 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a10ef63f1f442d974c27d5846dd638" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:49.647614 7274 raft_consensus.cc:2454] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:49.647588 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a10ef63f1f442d974c27d5846dd638" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:49.647727 7142 raft_consensus.cc:2454] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:49.647912 9616 leader_election.cc:302] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:49.648125 14361 raft_consensus.cc:2781] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:49.648170 14361 raft_consensus.cc:482] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:49.648211 14361 raft_consensus.cc:3037] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.649076 14361 raft_consensus.cc:504] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.649420 14361 leader_election.cc:288] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:49.649719 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a10ef63f1f442d974c27d5846dd638" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:49.649834 7274 raft_consensus.cc:3037] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.649999 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a10ef63f1f442d974c27d5846dd638" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:49.650096 7143 raft_consensus.cc:3037] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.650476 15237 raft_consensus.cc:482] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:49.650555 15237 raft_consensus.cc:504] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:49.650887 7274 raft_consensus.cc:2454] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:49.650911 15237 leader_election.cc:288] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:49.651162 7143 raft_consensus.cc:2454] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:49.651369 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a31b0f75c2514916a94205ce0289629e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:49.651512 9547 raft_consensus.cc:2454] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:49.651686 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a31b0f75c2514916a94205ce0289629e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:49.651757 9614 leader_election.cc:302] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:49.651822 6882 raft_consensus.cc:2454] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:49.651896 6946 leader_election.cc:302] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:49.652019 15494 raft_consensus.cc:2781] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:49.652102 15494 raft_consensus.cc:686] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:49.652258 15670 raft_consensus.cc:2781] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:49.652252 15494 consensus_queue.cc:237] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:49.652330 15670 raft_consensus.cc:482] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:49.652371 15670 raft_consensus.cc:3037] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.653267 15670 raft_consensus.cc:504] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:49.653625 15670 leader_election.cc:288] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:49.653939 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a31b0f75c2514916a94205ce0289629e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:49.654048 6882 raft_consensus.cc:3037] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.654026 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a31b0f75c2514916a94205ce0289629e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:49.654142 9549 raft_consensus.cc:3037] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:49.655104 9549 raft_consensus.cc:2454] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:49.655189 6882 raft_consensus.cc:2454] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:49.655428 6946 leader_election.cc:302] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:49.655620 15237 raft_consensus.cc:2781] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:49.655711 15237 raft_consensus.cc:686] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:49.655860 15237 consensus_queue.cc:237] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:49.656323 6738 catalog_manager.cc:5484] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:49.658162 6732 catalog_manager.cc:5484] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:49.677533 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:49.685458 6991 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:49.689751 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:49.695791 9527 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:49.700500 6860 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:49.703330 7255 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:49.712769 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:49.716996 9395 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:49.838791 6733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:49.845778 7143 raft_consensus.cc:1261] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:49.845902 7274 raft_consensus.cc:1261] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:49.846256 15494 consensus_queue.cc:1035] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:49.846637 15494 consensus_queue.cc:1035] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:49.850124 16531 tablet.cc:1722] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:49.852715 16531 tablet.cc:1620] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:49.856809 6732 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 07a10ef63f1f442d974c27d5846dd638 (table rebalancer_dynamic_table [id=2d023942a6174555a6b2a1633ff0e709]). Expected version 1 got 0
I20240426 09:49:49.860208 6735 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet 07a10ef63f1f442d974c27d5846dd638 (table rebalancer_dynamic_table [id=2d023942a6174555a6b2a1633ff0e709]). Expected version 1 got 0
I20240426 09:49:49.860458 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=2d023942a6174555a6b2a1633ff0e709] alter complete (version 1)
I20240426 09:49:49.862658 16532 tablet.cc:1722] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:49.862975 16532 tablet.cc:1620] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:49.863834 6732 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet 07a10ef63f1f442d974c27d5846dd638 (table rebalancer_dynamic_table [id=2d023942a6174555a6b2a1633ff0e709]). Expected version 1 got 0
I20240426 09:49:49.864312 6735 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 07a10ef63f1f442d974c27d5846dd638 (table rebalancer_dynamic_table [id=2d023942a6174555a6b2a1633ff0e709]). Expected version 1 got 0
I20240426 09:49:49.866848 16533 tablet.cc:1722] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:49.867170 16533 tablet.cc:1620] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:49.880137 9395 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:49.883984 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:49.888595 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:49.894284 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:49.895506 6991 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:49.899200 6860 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:49.899655 7255 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:49.930470 9527 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:49.962301 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:49.962703 16265 consensus_queue.cc:1035] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:49.971448 16531 tablet.cc:1722] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:49.971769 16531 tablet.cc:1620] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:49.972592 16532 tablet.cc:1722] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:49.972885 16532 tablet.cc:1620] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:49.974933 16112 consensus_queue.cc:1035] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:49.978890 16533 tablet.cc:1722] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:49.979260 16533 tablet.cc:1620] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:49.981832 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=2d023942a6174555a6b2a1633ff0e709] alter complete (version 2)
I20240426 09:49:49.987185 16552 ts_tablet_manager.cc:902] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:49.989028 16552 tablet_copy_client.cc:308] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:49:49.989502 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 956f89ba28c54473964cc877c26580f0 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:49245)
I20240426 09:49:49.989603 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 956f89ba28c54473964cc877c26580f0 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:49245: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-956f89ba28c54473964cc877c26580f0
I20240426 09:49:49.990494 7164 tablet_copy_source_session.cc:205] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:49.991344 16552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 956f89ba28c54473964cc877c26580f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:49.993556 16552 tablet_copy_client.cc:783] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:49.993717 16552 tablet_copy_client.cc:647] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:49.994746 16552 tablet_copy_client.cc:523] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:49.996075 16552 tablet_bootstrap.cc:492] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:49.998708 6880 consensus_queue.cc:237] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:50.000232 16277 consensus_queue.cc:1035] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.000281 7274 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:50.000514 9669 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:50.001041 15178 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:50.003816 16277 consensus_queue.cc:1035] T 635293adcd7b45858935a65e4ba58f24 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.004606 15178 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:50.007663 16574 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:50.010448 9669 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:50.013742 7274 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:50.015651 6732 catalog_manager.cc:5484] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:50.018880 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fe43b0765d9401a88c8e9a028ab3ef4 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:50.019219 16580 tablet_replica.cc:330] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:50.019459 16580 raft_consensus.cc:2229] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:50.019738 16580 raft_consensus.cc:2258] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:50.021862 16580 ts_tablet_manager.cc:1875] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:50.024303 16580 ts_tablet_manager.cc:1888] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:50.024374 16580 log.cc:1196] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/5fe43b0765d9401a88c8e9a028ab3ef4
I20240426 09:49:50.024864 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 5fe43b0765d9401a88c8e9a028ab3ef4 (table rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]) successfully deleted
I20240426 09:49:50.029477 16552 tablet_bootstrap.cc:492] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:50.029752 16552 tablet_bootstrap.cc:492] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:50.029927 16552 ts_tablet_manager.cc:1367] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.034s user 0.025s sys 0.000s
I20240426 09:49:50.030372 16552 raft_consensus.cc:348] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:50.030505 16552 raft_consensus.cc:729] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.030656 16552 consensus_queue.cc:260] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:50.031755 16552 ts_tablet_manager.cc:1398] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20240426 09:49:50.032205 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-956f89ba28c54473964cc877c26580f0 received from {username='slave'} at 127.0.99.135:49245
I20240426 09:49:50.032272 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-956f89ba28c54473964cc877c26580f0 on tablet 956f89ba28c54473964cc877c26580f0 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:50.084156 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:50.092097 16531 tablet.cc:1722] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:50.092698 16533 tablet.cc:1722] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:50.094135 16532 tablet.cc:1722] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:50.100391 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=2d023942a6174555a6b2a1633ff0e709] alter complete (version 3)
I20240426 09:49:50.138193 6860 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:50.141551 6991 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:50.144861 9395 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:50.155081 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:50.159310 9527 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:50.161335 15670 consensus_queue.cc:1035] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.165400 15237 consensus_queue.cc:1035] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.170224 7255 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:50.172011 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:50.182309 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:50.203797 6740 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:50.203982 6740 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:50.206053 6740 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 07a10ef63f1f442d974c27d5846dd638
I20240426 09:49:50.206650 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a10ef63f1f442d974c27d5846dd638 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:50 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:50.206692 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a10ef63f1f442d974c27d5846dd638 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:50 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:50.206754 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a10ef63f1f442d974c27d5846dd638 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:50 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:50.207098 16580 tablet_replica.cc:330] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:50.207265 16580 raft_consensus.cc:2229] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:50.207386 16580 raft_consensus.cc:2258] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:50.208002 16580 ts_tablet_manager.cc:1875] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:50.210065 16580 ts_tablet_manager.cc:1888] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:50.210152 16580 log.cc:1196] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/07a10ef63f1f442d974c27d5846dd638
I20240426 09:49:50.210395 16580 ts_tablet_manager.cc:1909] T 07a10ef63f1f442d974c27d5846dd638 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:50.211428 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 07a10ef63f1f442d974c27d5846dd638 (table rebalancer_dynamic_table_new_name [id=2d023942a6174555a6b2a1633ff0e709]) successfully deleted
I20240426 09:49:50.214954 16638 tablet_replica.cc:330] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:50.215210 16638 raft_consensus.cc:2229] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:50.215397 16638 raft_consensus.cc:2258] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:50.215948 16638 ts_tablet_manager.cc:1875] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:50.216908 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:50.218384 16638 ts_tablet_manager.cc:1888] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:50.218395 16637 tablet_replica.cc:330] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:50.218467 16638 log.cc:1196] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/07a10ef63f1f442d974c27d5846dd638
I20240426 09:49:50.218569 16637 raft_consensus.cc:2229] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:50.218829 16638 ts_tablet_manager.cc:1909] T 07a10ef63f1f442d974c27d5846dd638 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:50.218830 16637 raft_consensus.cc:2258] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:50.219525 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 07a10ef63f1f442d974c27d5846dd638 (table rebalancer_dynamic_table_new_name [id=2d023942a6174555a6b2a1633ff0e709]) successfully deleted
I20240426 09:49:50.219674 16637 ts_tablet_manager.cc:1875] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:50.222178 16637 ts_tablet_manager.cc:1888] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:50.222299 16637 log.cc:1196] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/07a10ef63f1f442d974c27d5846dd638
I20240426 09:49:50.222697 16637 ts_tablet_manager.cc:1909] T 07a10ef63f1f442d974c27d5846dd638 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:50.224018 9395 tablet_service.cc:1467] Processing CreateTablet for tablet e110e59d534c4bbd9fbceae078bb8286 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.224308 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e110e59d534c4bbd9fbceae078bb8286. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.224341 6860 tablet_service.cc:1467] Processing CreateTablet for tablet e110e59d534c4bbd9fbceae078bb8286 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.224426 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 0d04a7ec2c62437b879de204b5c885e9 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.224503 7255 tablet_service.cc:1467] Processing CreateTablet for tablet e110e59d534c4bbd9fbceae078bb8286 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.224594 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e110e59d534c4bbd9fbceae078bb8286. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.224716 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e110e59d534c4bbd9fbceae078bb8286. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.224910 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d04a7ec2c62437b879de204b5c885e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.226735 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 0d04a7ec2c62437b879de204b5c885e9 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.227023 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d04a7ec2c62437b879de204b5c885e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.227674 16639 tablet_bootstrap.cc:492] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:50.228554 16639 tablet_bootstrap.cc:654] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.229312 16640 tablet_bootstrap.cc:492] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:50.229879 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 07a10ef63f1f442d974c27d5846dd638 (table rebalancer_dynamic_table_new_name [id=2d023942a6174555a6b2a1633ff0e709]) successfully deleted
I20240426 09:49:50.230479 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 0d04a7ec2c62437b879de204b5c885e9 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.230525 7122 tablet_service.cc:1467] Processing CreateTablet for tablet d60546e3101f4cbd9bc137d73ed5dea8 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.230693 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d04a7ec2c62437b879de204b5c885e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.230729 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d60546e3101f4cbd9bc137d73ed5dea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.231098 6993 tablet_service.cc:1467] Processing CreateTablet for tablet d60546e3101f4cbd9bc137d73ed5dea8 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.231285 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d60546e3101f4cbd9bc137d73ed5dea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.232676 16643 tablet_bootstrap.cc:492] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:50.233261 9661 tablet_service.cc:1467] Processing CreateTablet for tablet d60546e3101f4cbd9bc137d73ed5dea8 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.233544 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d60546e3101f4cbd9bc137d73ed5dea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.233852 16639 tablet_bootstrap.cc:492] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:50.233935 16639 ts_tablet_manager.cc:1367] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:50.234028 16644 tablet_bootstrap.cc:492] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:50.235098 16644 tablet_bootstrap.cc:654] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.235781 16640 tablet_bootstrap.cc:654] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.236079 16644 tablet_bootstrap.cc:492] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:50.236156 16644 ts_tablet_manager.cc:1367] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:50.236596 16643 tablet_bootstrap.cc:654] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.236574 16644 raft_consensus.cc:348] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.236696 16644 raft_consensus.cc:374] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.236640 16639 raft_consensus.cc:348] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:50.236733 16639 raft_consensus.cc:374] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.236726 16644 raft_consensus.cc:729] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.236763 16639 raft_consensus.cc:729] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.236872 16639 consensus_queue.cc:260] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:50.236871 16644 consensus_queue.cc:260] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.237120 16644 ts_tablet_manager.cc:1398] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:50.237186 16639 ts_tablet_manager.cc:1398] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:50.237269 16644 tablet_bootstrap.cc:492] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:50.237367 16641 tablet_bootstrap.cc:492] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:50.238095 16644 tablet_bootstrap.cc:654] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.238871 16646 tablet_bootstrap.cc:492] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:50.239434 16644 tablet_bootstrap.cc:492] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:50.239523 16644 ts_tablet_manager.cc:1367] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:50.239606 16641 tablet_bootstrap.cc:654] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.239962 16644 raft_consensus.cc:348] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.240057 16644 raft_consensus.cc:374] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.240085 16644 raft_consensus.cc:729] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.240206 16644 consensus_queue.cc:260] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.240459 16644 ts_tablet_manager.cc:1398] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:50.241587 16646 tablet_bootstrap.cc:654] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.243284 16641 tablet_bootstrap.cc:492] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:50.243391 16641 ts_tablet_manager.cc:1367] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:49:50.243837 16641 raft_consensus.cc:348] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:50.243933 16641 raft_consensus.cc:374] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.243965 16641 raft_consensus.cc:729] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.244146 16641 consensus_queue.cc:260] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:50.244484 16641 ts_tablet_manager.cc:1398] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:50.244835 16641 tablet_bootstrap.cc:492] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:50.247146 16640 tablet_bootstrap.cc:492] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:50.247256 16640 ts_tablet_manager.cc:1367] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240426 09:49:50.247637 16640 raft_consensus.cc:348] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:50.247750 16640 raft_consensus.cc:374] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.247807 16640 raft_consensus.cc:729] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.247984 16640 consensus_queue.cc:260] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:50.248378 16640 ts_tablet_manager.cc:1398] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:50.248678 16640 tablet_bootstrap.cc:492] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:50.250427 16643 tablet_bootstrap.cc:492] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:50.250602 16643 ts_tablet_manager.cc:1367] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240426 09:49:50.250619 16640 tablet_bootstrap.cc:654] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.251125 16643 raft_consensus.cc:348] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.251308 16643 raft_consensus.cc:374] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.251384 16643 raft_consensus.cc:729] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.251538 16646 tablet_bootstrap.cc:492] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:50.251636 16646 ts_tablet_manager.cc:1367] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:50.251694 16640 tablet_bootstrap.cc:492] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:50.251569 16643 consensus_queue.cc:260] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.251770 16640 ts_tablet_manager.cc:1367] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:50.251943 16643 ts_tablet_manager.cc:1398] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:50.252090 16646 raft_consensus.cc:348] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.252187 16646 raft_consensus.cc:374] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.252215 16646 raft_consensus.cc:729] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.252163 16640 raft_consensus.cc:348] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.252260 16640 raft_consensus.cc:374] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.252295 16640 raft_consensus.cc:729] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.252319 16646 consensus_queue.cc:260] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.252398 16640 consensus_queue.cc:260] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.252578 16646 ts_tablet_manager.cc:1398] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:50.252691 16640 ts_tablet_manager.cc:1398] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:50.258437 16641 tablet_bootstrap.cc:654] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.259721 16641 tablet_bootstrap.cc:492] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:50.259820 16641 ts_tablet_manager.cc:1367] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240426 09:49:50.260264 16641 raft_consensus.cc:348] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.260360 16641 raft_consensus.cc:374] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.260380 16641 raft_consensus.cc:729] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.260499 16641 consensus_queue.cc:260] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.260775 16641 ts_tablet_manager.cc:1398] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:50.269454 7143 consensus_queue.cc:237] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:50.270895 9669 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:50.270892 7274 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:50.271304 16277 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:50.271649 16576 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:50.273483 16277 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:50.273947 9669 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:50.274276 7274 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:50.276813 6732 catalog_manager.cc:5484] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:50.280201 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet cd377032e7cd4b4998424f8685ad5b85 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:50.280550 16677 tablet_replica.cc:330] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:50.280892 16677 raft_consensus.cc:2229] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:50.281095 16677 raft_consensus.cc:2258] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:50.281669 16677 ts_tablet_manager.cc:1875] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:50.284670 16677 ts_tablet_manager.cc:1888] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:50.284780 16677 log.cc:1196] T cd377032e7cd4b4998424f8685ad5b85 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/cd377032e7cd4b4998424f8685ad5b85
I20240426 09:49:50.285343 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet cd377032e7cd4b4998424f8685ad5b85 (table rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]) successfully deleted
I20240426 09:49:50.308228 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:50.314498 9266 tablet_service.cc:1467] Processing CreateTablet for tablet e01872a7831148dfb71df2c683401b6e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=696df28e329941d386415d2f66bdc5d9]), partition=
I20240426 09:49:50.314661 6993 tablet_service.cc:1467] Processing CreateTablet for tablet e01872a7831148dfb71df2c683401b6e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=696df28e329941d386415d2f66bdc5d9]), partition=
I20240426 09:49:50.314675 9661 tablet_service.cc:1467] Processing CreateTablet for tablet e01872a7831148dfb71df2c683401b6e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=696df28e329941d386415d2f66bdc5d9]), partition=
I20240426 09:49:50.314810 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01872a7831148dfb71df2c683401b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.314937 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01872a7831148dfb71df2c683401b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.314970 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01872a7831148dfb71df2c683401b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.317327 16644 tablet_bootstrap.cc:492] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:50.317577 16646 tablet_bootstrap.cc:492] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:50.318324 16644 tablet_bootstrap.cc:654] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.318548 16646 tablet_bootstrap.cc:654] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.319850 16644 tablet_bootstrap.cc:492] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:50.319942 16644 ts_tablet_manager.cc:1367] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:50.320361 16646 tablet_bootstrap.cc:492] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:50.320430 16646 ts_tablet_manager.cc:1367] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:50.320391 16644 raft_consensus.cc:348] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.320523 16644 raft_consensus.cc:374] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.320557 16644 raft_consensus.cc:729] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.320675 16644 consensus_queue.cc:260] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.320936 16644 ts_tablet_manager.cc:1398] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:50.320863 16646 raft_consensus.cc:348] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.320981 16646 raft_consensus.cc:374] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.321017 16646 raft_consensus.cc:729] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.321134 16646 consensus_queue.cc:260] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.321449 16646 ts_tablet_manager.cc:1398] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:50.334887 16695 tablet_bootstrap.cc:492] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:50.337536 16695 tablet_bootstrap.cc:654] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:50.343926 16695 tablet_bootstrap.cc:492] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:50.344092 16695 ts_tablet_manager.cc:1367] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20240426 09:49:50.345039 16695 raft_consensus.cc:348] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.345360 16695 raft_consensus.cc:374] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:50.345465 16695 raft_consensus.cc:729] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:50.345759 16695 consensus_queue.cc:260] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.346997 16695 ts_tablet_manager.cc:1398] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:50.371603 16547 raft_consensus.cc:482] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:50.371735 16547 raft_consensus.cc:504] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.372249 16547 leader_election.cc:288] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:50.372687 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d60546e3101f4cbd9bc137d73ed5dea8" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:50.372825 7143 raft_consensus.cc:2454] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:50.372798 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d60546e3101f4cbd9bc137d73ed5dea8" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:50.372946 7013 raft_consensus.cc:2454] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:50.373648 9614 leader_election.cc:302] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:50.373836 16547 raft_consensus.cc:2781] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:50.373896 16547 raft_consensus.cc:482] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:50.373946 16547 raft_consensus.cc:3037] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.375120 16547 raft_consensus.cc:504] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.375461 16547 leader_election.cc:288] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:50.376684 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d60546e3101f4cbd9bc137d73ed5dea8" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:50.376813 7143 raft_consensus.cc:3037] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.377900 7143 raft_consensus.cc:2454] T d60546e3101f4cbd9bc137d73ed5dea8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:50.379084 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d60546e3101f4cbd9bc137d73ed5dea8" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:50.379205 7013 raft_consensus.cc:3037] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.380316 7013 raft_consensus.cc:2454] T d60546e3101f4cbd9bc137d73ed5dea8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:50.380977 9614 leader_election.cc:302] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:50.381297 16547 raft_consensus.cc:2781] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:50.381384 16547 raft_consensus.cc:686] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:50.381512 16547 consensus_queue.cc:237] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.384970 6737 catalog_manager.cc:5484] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:50.406970 15237 raft_consensus.cc:482] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:50.407094 15237 raft_consensus.cc:504] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.407478 15237 leader_election.cc:288] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:50.407780 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01872a7831148dfb71df2c683401b6e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:50.407809 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01872a7831148dfb71df2c683401b6e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:50.407908 9669 raft_consensus.cc:2454] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:50.407953 9285 raft_consensus.cc:2454] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:50.408206 6946 leader_election.cc:302] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:50.408540 15237 raft_consensus.cc:2781] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:50.408596 15237 raft_consensus.cc:482] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:50.408636 15237 raft_consensus.cc:3037] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.409644 15237 raft_consensus.cc:504] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.409982 15237 leader_election.cc:288] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:50.410319 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01872a7831148dfb71df2c683401b6e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:50.410348 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01872a7831148dfb71df2c683401b6e" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:50.410439 9285 raft_consensus.cc:3037] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.410436 9669 raft_consensus.cc:3037] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.411343 9285 raft_consensus.cc:2454] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:50.411409 9669 raft_consensus.cc:2454] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:50.411644 6946 leader_election.cc:302] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:50.411779 15237 raft_consensus.cc:2781] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:50.411870 15237 raft_consensus.cc:686] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:50.411996 15237 consensus_queue.cc:237] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.414109 6738 catalog_manager.cc:5484] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:50.414292 16574 raft_consensus.cc:482] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:50.414388 16574 raft_consensus.cc:504] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:50.414840 16574 leader_election.cc:288] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:50.415848 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e110e59d534c4bbd9fbceae078bb8286" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:50.415987 7274 raft_consensus.cc:2454] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:50.416206 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e110e59d534c4bbd9fbceae078bb8286" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:50.416357 9417 raft_consensus.cc:2454] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:50.416569 6817 leader_election.cc:302] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:50.416730 16574 raft_consensus.cc:2781] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:50.416771 16574 raft_consensus.cc:482] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:50.416810 16574 raft_consensus.cc:3037] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.417572 16574 raft_consensus.cc:504] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:50.417870 16574 leader_election.cc:288] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:50.418175 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e110e59d534c4bbd9fbceae078bb8286" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:50.418279 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e110e59d534c4bbd9fbceae078bb8286" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:50.418319 7274 raft_consensus.cc:3037] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.418373 9417 raft_consensus.cc:3037] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.419306 9417 raft_consensus.cc:2454] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:50.419306 7274 raft_consensus.cc:2454] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:50.419584 6817 leader_election.cc:302] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:50.419745 16574 raft_consensus.cc:2781] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:50.419835 16574 raft_consensus.cc:686] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:50.419960 16574 consensus_queue.cc:237] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:50.422128 6738 catalog_manager.cc:5484] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:50.447852 16574 raft_consensus.cc:482] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:50.447949 16574 raft_consensus.cc:504] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.448604 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d04a7ec2c62437b879de204b5c885e9" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:50.448734 7274 raft_consensus.cc:2454] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:50.448308 16574 leader_election.cc:288] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:50.449030 6817 leader_election.cc:302] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:50.449180 16574 raft_consensus.cc:2781] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:50.449232 16574 raft_consensus.cc:482] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:50.449265 16574 raft_consensus.cc:3037] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.449800 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d04a7ec2c62437b879de204b5c885e9" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:50.449921 7013 raft_consensus.cc:2454] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:50.450332 16574 raft_consensus.cc:504] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.450691 16574 leader_election.cc:288] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:50.450932 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d04a7ec2c62437b879de204b5c885e9" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:50.451048 7274 raft_consensus.cc:3037] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.451007 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d04a7ec2c62437b879de204b5c885e9" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:50.451124 7013 raft_consensus.cc:3037] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:50.451855 7274 raft_consensus.cc:2454] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:50.451898 7013 raft_consensus.cc:2454] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:50.452122 6817 leader_election.cc:302] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:50.452268 16574 raft_consensus.cc:2781] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:50.452414 16574 raft_consensus.cc:686] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:50.452549 16574 consensus_queue.cc:237] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:50.454461 6738 catalog_manager.cc:5484] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:50.466197 9547 raft_consensus.cc:1203] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:50.473438 6993 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:50.477481 9395 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:50.479553 6862 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:50.481714 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:50.493993 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:50.504897 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:50.512766 9527 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:50.524426 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:50.529174 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:50.531205 9669 raft_consensus.cc:1261] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:50.531198 9285 raft_consensus.cc:1261] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:50.531579 15670 consensus_queue.cc:1035] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.532251 15670 consensus_queue.cc:1035] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.535415 6738 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet e01872a7831148dfb71df2c683401b6e (table rebalancer_dynamic_table [id=696df28e329941d386415d2f66bdc5d9]). Expected version 1 got 0
I20240426 09:49:50.536510 6740 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet e01872a7831148dfb71df2c683401b6e (table rebalancer_dynamic_table [id=696df28e329941d386415d2f66bdc5d9]). Expected version 1 got 0
I20240426 09:49:50.542102 6740 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet e01872a7831148dfb71df2c683401b6e (table rebalancer_dynamic_table [id=696df28e329941d386415d2f66bdc5d9]). Expected version 1 got 0
I20240426 09:49:50.544777 16531 tablet.cc:1722] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:50.545092 16531 tablet.cc:1620] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:50.548933 16739 tablet.cc:1722] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:50.549607 16739 tablet.cc:1620] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:50.550596 6740 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=696df28e329941d386415d2f66bdc5d9] alter complete (version 1)
I20240426 09:49:50.557426 16740 tablet.cc:1722] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:50.557827 16740 tablet.cc:1620] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:50.594708 9669 consensus_queue.cc:237] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:50.595938 7274 raft_consensus.cc:1261] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:50.596338 14361 consensus_queue.cc:1035] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:50.596390 9285 raft_consensus.cc:1261] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:50.596823 14361 consensus_queue.cc:1035] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:50.598479 16547 raft_consensus.cc:2932] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:50.600081 7274 raft_consensus.cc:2932] T 2c41aab70e4a4b4e86cf095575cd8805 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:50.602094 6740 catalog_manager.cc:5484] T 2c41aab70e4a4b4e86cf095575cd8805 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:50.602077 9285 raft_consensus.cc:2932] T 2c41aab70e4a4b4e86cf095575cd8805 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:50.604704 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c41aab70e4a4b4e86cf095575cd8805 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:50.605041 16759 tablet_replica.cc:330] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:50.605255 16759 raft_consensus.cc:2229] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:50.605562 16759 raft_consensus.cc:2258] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:50.606241 16759 ts_tablet_manager.cc:1875] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:50.608883 16759 ts_tablet_manager.cc:1888] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:50.608960 16759 log.cc:1196] T 2c41aab70e4a4b4e86cf095575cd8805 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/2c41aab70e4a4b4e86cf095575cd8805
I20240426 09:49:50.609489 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 2c41aab70e4a4b4e86cf095575cd8805 (table rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]) successfully deleted
I20240426 09:49:50.638599 9669 consensus_queue.cc:237] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:50.640055 9547 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:50.640039 6880 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:50.640456 16791 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:49:50.640666 9614 consensus_peers.cc:485] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53765d0ffcc24bb98aadc0cd0fe44af3. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:50.640866 16791 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:50.642999 16547 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:50.643416 6882 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:50.646062 9547 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:50.646322 6740 catalog_manager.cc:5484] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:50.656634 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:50.662266 16739 tablet.cc:1722] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:50.662601 16739 tablet.cc:1620] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:50.662909 16740 tablet.cc:1722] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:50.663194 16740 tablet.cc:1620] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:50.663432 16531 tablet.cc:1722] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:50.663729 16531 tablet.cc:1620] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:50.668385 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=696df28e329941d386415d2f66bdc5d9] alter complete (version 2)
I20240426 09:49:50.761255 16791 consensus_queue.cc:1035] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.772164 6733 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:50.775022 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:50.775712 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:50.775846 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:50.778219 16791 consensus_queue.cc:1035] T d60546e3101f4cbd9bc137d73ed5dea8 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.779547 6862 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:50.784407 16739 tablet.cc:1722] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:50.795163 16531 tablet.cc:1722] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:50.797405 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=696df28e329941d386415d2f66bdc5d9] alter complete (version 3)
I20240426 09:49:50.803316 16740 tablet.cc:1722] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:50.804267 9395 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:50.809931 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:50.819392 9527 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:50.821045 6993 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:50.867597 16574 consensus_queue.cc:1035] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.870692 16574 consensus_queue.cc:1035] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.902262 6739 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:50.902412 6739 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:50.904673 6739 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet e01872a7831148dfb71df2c683401b6e
I20240426 09:49:50.905241 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet e01872a7831148dfb71df2c683401b6e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:50 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:50.905256 9266 tablet_service.cc:1514] Processing DeleteTablet for tablet e01872a7831148dfb71df2c683401b6e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:50 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:50.905323 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet e01872a7831148dfb71df2c683401b6e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:50 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:50.905826 16892 tablet_replica.cc:330] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:50.906009 16891 tablet_replica.cc:330] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:50.906044 16892 raft_consensus.cc:2229] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:50.906153 16891 raft_consensus.cc:2229] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:50.906176 16892 raft_consensus.cc:2258] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:50.906281 16891 raft_consensus.cc:2258] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:50.907008 16891 ts_tablet_manager.cc:1875] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:50.907867 16892 ts_tablet_manager.cc:1875] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:50.911545 16892 ts_tablet_manager.cc:1888] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:50.911731 16892 log.cc:1196] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/e01872a7831148dfb71df2c683401b6e
I20240426 09:49:50.912091 16892 ts_tablet_manager.cc:1909] T e01872a7831148dfb71df2c683401b6e P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:50.912668 16891 ts_tablet_manager.cc:1888] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:50.912794 16891 log.cc:1196] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/e01872a7831148dfb71df2c683401b6e
I20240426 09:49:50.912981 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet e01872a7831148dfb71df2c683401b6e (table rebalancer_dynamic_table_new_name [id=696df28e329941d386415d2f66bdc5d9]) successfully deleted
I20240426 09:49:50.913183 16891 ts_tablet_manager.cc:1909] T e01872a7831148dfb71df2c683401b6e P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:50.913906 16893 tablet_replica.cc:330] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:50.914014 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet e01872a7831148dfb71df2c683401b6e (table rebalancer_dynamic_table_new_name [id=696df28e329941d386415d2f66bdc5d9]) successfully deleted
I20240426 09:49:50.914165 16893 raft_consensus.cc:2229] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:50.914383 16893 raft_consensus.cc:2258] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:50.915068 16893 ts_tablet_manager.cc:1875] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:50.917464 16893 ts_tablet_manager.cc:1888] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:50.917636 16893 log.cc:1196] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/e01872a7831148dfb71df2c683401b6e
I20240426 09:49:50.919253 16893 ts_tablet_manager.cc:1909] T e01872a7831148dfb71df2c683401b6e P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:50.920922 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet e01872a7831148dfb71df2c683401b6e (table rebalancer_dynamic_table_new_name [id=696df28e329941d386415d2f66bdc5d9]) successfully deleted
I20240426 09:49:50.976079 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:50.980192 9395 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:50.983446 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:50.985347 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:50.986730 7122 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:50.991518 16574 consensus_queue.cc:1035] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:50.992837 9527 tablet_service.cc:1467] Processing CreateTablet for tablet b017fe28f5f241af8955642664afaaa8 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.993153 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b017fe28f5f241af8955642664afaaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.993753 7122 tablet_service.cc:1467] Processing CreateTablet for tablet b017fe28f5f241af8955642664afaaa8 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.994001 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b017fe28f5f241af8955642664afaaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.995544 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 3d5f900639e2494b97e13f3703536f42 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.996680 9266 tablet_service.cc:1467] Processing CreateTablet for tablet 3d5f900639e2494b97e13f3703536f42 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.996934 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d5f900639e2494b97e13f3703536f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.997701 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 1d720846739e49f888da92befe06a309 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:50.997921 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d720846739e49f888da92befe06a309. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:50.998281 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:50.998631 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d5f900639e2494b97e13f3703536f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.003058 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 1d720846739e49f888da92befe06a309 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.003347 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d720846739e49f888da92befe06a309. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.004897 9395 tablet_service.cc:1467] Processing CreateTablet for tablet b017fe28f5f241af8955642664afaaa8 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.004983 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 1d720846739e49f888da92befe06a309 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.005156 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b017fe28f5f241af8955642664afaaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.005905 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d720846739e49f888da92befe06a309. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.004897 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 3d5f900639e2494b97e13f3703536f42 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.007730 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d5f900639e2494b97e13f3703536f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.013567 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:51.021390 7253 tablet_service.cc:1467] Processing CreateTablet for tablet c5092f1e433f4f1cb27a169333aaecda (DEFAULT_TABLE table=rebalancer_dynamic_table [id=28697c769c2d411798cc40202e6c3862]), partition=
I20240426 09:49:51.021651 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5092f1e433f4f1cb27a169333aaecda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.021803 6993 tablet_service.cc:1467] Processing CreateTablet for tablet c5092f1e433f4f1cb27a169333aaecda (DEFAULT_TABLE table=rebalancer_dynamic_table [id=28697c769c2d411798cc40202e6c3862]), partition=
I20240426 09:49:51.022027 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5092f1e433f4f1cb27a169333aaecda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.024183 7122 tablet_service.cc:1467] Processing CreateTablet for tablet c5092f1e433f4f1cb27a169333aaecda (DEFAULT_TABLE table=rebalancer_dynamic_table [id=28697c769c2d411798cc40202e6c3862]), partition=
I20240426 09:49:51.024389 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5092f1e433f4f1cb27a169333aaecda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.024814 9529 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:51.028151 6862 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:51.030169 16913 tablet_bootstrap.cc:492] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:51.031087 16913 tablet_bootstrap.cc:654] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.037452 16910 tablet_bootstrap.cc:492] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:51.037739 16912 tablet_bootstrap.cc:492] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:51.039732 16910 tablet_bootstrap.cc:654] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.040923 16912 tablet_bootstrap.cc:654] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.043223 16912 tablet_bootstrap.cc:492] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:51.043318 16912 ts_tablet_manager.cc:1367] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:51.043812 16912 raft_consensus.cc:348] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.043918 16912 raft_consensus.cc:374] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.043946 16912 raft_consensus.cc:729] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.044071 16912 consensus_queue.cc:260] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.044322 16912 ts_tablet_manager.cc:1398] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.046042 16574 consensus_queue.cc:1035] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:51.047659 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:51.057713 16909 tablet_bootstrap.cc:492] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:51.058357 16908 tablet_bootstrap.cc:492] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:51.060496 16909 tablet_bootstrap.cc:654] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.063831 16908 tablet_bootstrap.cc:654] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.064414 16910 tablet_bootstrap.cc:492] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:51.064534 16910 ts_tablet_manager.cc:1367] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.003s
I20240426 09:49:51.065011 16910 raft_consensus.cc:348] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.065142 16910 raft_consensus.cc:374] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.065172 16910 raft_consensus.cc:729] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.065299 16910 consensus_queue.cc:260] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.065567 16910 ts_tablet_manager.cc:1398] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:51.065725 16910 tablet_bootstrap.cc:492] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:51.066605 16910 tablet_bootstrap.cc:654] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.068145 16910 tablet_bootstrap.cc:492] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:51.068240 16910 ts_tablet_manager.cc:1367] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:51.068646 16910 raft_consensus.cc:348] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.068738 16910 raft_consensus.cc:374] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.068763 16910 raft_consensus.cc:729] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.068862 16910 consensus_queue.cc:260] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.069080 16910 ts_tablet_manager.cc:1398] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:51.069226 16910 tablet_bootstrap.cc:492] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:51.070098 16910 tablet_bootstrap.cc:654] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.071208 16914 tablet_bootstrap.cc:492] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:51.071851 16907 tablet_bootstrap.cc:492] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:51.075003 16909 tablet_bootstrap.cc:492] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:51.075165 16909 ts_tablet_manager.cc:1367] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.001s
I20240426 09:49:51.075255 16907 tablet_bootstrap.cc:654] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.076366 16914 tablet_bootstrap.cc:654] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.076298 16909 raft_consensus.cc:348] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.076516 16909 raft_consensus.cc:374] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.076604 16909 raft_consensus.cc:729] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.076794 16909 consensus_queue.cc:260] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.077354 16909 ts_tablet_manager.cc:1398] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:51.078162 16908 tablet_bootstrap.cc:492] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:51.078298 16908 ts_tablet_manager.cc:1367] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240426 09:49:51.078840 16908 raft_consensus.cc:348] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.078950 16910 tablet_bootstrap.cc:492] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:51.078990 16908 raft_consensus.cc:374] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.079069 16910 ts_tablet_manager.cc:1367] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240426 09:49:51.079027 16908 raft_consensus.cc:729] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.080008 16908 consensus_queue.cc:260] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.080024 16910 raft_consensus.cc:348] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.080116 16910 raft_consensus.cc:374] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.080155 16910 raft_consensus.cc:729] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.080284 16910 consensus_queue.cc:260] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.080487 16908 ts_tablet_manager.cc:1398] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:51.080567 16910 ts_tablet_manager.cc:1398] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:51.081192 16908 tablet_bootstrap.cc:492] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:51.085687 16908 tablet_bootstrap.cc:654] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.088776 16914 tablet_bootstrap.cc:492] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:51.090041 16907 tablet_bootstrap.cc:492] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:51.090150 16907 ts_tablet_manager.cc:1367] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240426 09:49:51.090646 16907 raft_consensus.cc:348] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.090795 16907 raft_consensus.cc:374] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.091086 16914 ts_tablet_manager.cc:1367] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240426 09:49:51.091645 16914 raft_consensus.cc:348] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:51.091761 16914 raft_consensus.cc:374] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.091831 16907 raft_consensus.cc:729] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.091972 16914 raft_consensus.cc:729] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.092149 16907 consensus_queue.cc:260] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.092414 16907 ts_tablet_manager.cc:1398] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:51.092584 16907 tablet_bootstrap.cc:492] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:51.093384 16907 tablet_bootstrap.cc:654] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.094415 16914 consensus_queue.cc:260] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:51.095717 16914 ts_tablet_manager.cc:1398] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240426 09:49:51.097044 16908 tablet_bootstrap.cc:492] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:51.097198 16908 ts_tablet_manager.cc:1367] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.001s
I20240426 09:49:51.097710 16908 raft_consensus.cc:348] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.097816 16908 raft_consensus.cc:374] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.097841 16908 raft_consensus.cc:729] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.098014 16908 consensus_queue.cc:260] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.098569 16908 ts_tablet_manager.cc:1398] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.099033 16908 tablet_bootstrap.cc:492] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:51.101053 16908 tablet_bootstrap.cc:654] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.102425 16907 tablet_bootstrap.cc:492] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:51.102710 16907 ts_tablet_manager.cc:1367] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20240426 09:49:51.103180 16913 tablet_bootstrap.cc:492] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:51.103282 16913 ts_tablet_manager.cc:1367] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.073s user 0.001s sys 0.000s
I20240426 09:49:51.103633 16907 raft_consensus.cc:348] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.103756 16907 raft_consensus.cc:374] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.103721 16913 raft_consensus.cc:348] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:51.103814 16907 raft_consensus.cc:729] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.103863 16913 raft_consensus.cc:374] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.103909 16913 raft_consensus.cc:729] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.104040 16913 consensus_queue.cc:260] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:51.104142 16907 consensus_queue.cc:260] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.104310 16913 ts_tablet_manager.cc:1398] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.104480 16907 ts_tablet_manager.cc:1398] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240426 09:49:51.105396 16908 tablet_bootstrap.cc:492] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:51.105520 16908 ts_tablet_manager.cc:1367] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:51.105924 16908 raft_consensus.cc:348] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:51.106073 16908 raft_consensus.cc:374] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.106137 16908 raft_consensus.cc:729] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.106506 16908 consensus_queue.cc:260] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:51.106835 16908 ts_tablet_manager.cc:1398] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.119164 16934 ts_tablet_manager.cc:902] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:51.119886 16934 tablet_copy_client.cc:308] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:51.120635 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 53765d0ffcc24bb98aadc0cd0fe44af3 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:48359)
I20240426 09:49:51.120744 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 53765d0ffcc24bb98aadc0cd0fe44af3 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:48359: session id = 1aaf8befba984e199e96f53f46b50441-53765d0ffcc24bb98aadc0cd0fe44af3
I20240426 09:49:51.121520 9685 tablet_copy_source_session.cc:205] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:51.123034 16934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53765d0ffcc24bb98aadc0cd0fe44af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.126369 16934 tablet_copy_client.cc:783] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:51.126567 16934 tablet_copy_client.cc:647] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:51.127801 16934 tablet_copy_client.cc:523] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:51.129295 16934 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:51.134116 16934 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:51.134348 16934 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:49:51.134505 16934 ts_tablet_manager.cc:1367] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240426 09:49:51.134932 16934 raft_consensus.cc:348] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.135030 16934 raft_consensus.cc:729] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.135172 16934 consensus_queue.cc:260] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.135445 16934 ts_tablet_manager.cc:1398] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:51.135933 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-53765d0ffcc24bb98aadc0cd0fe44af3 received from {username='slave'} at 127.0.99.131:48359
I20240426 09:49:51.136008 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-53765d0ffcc24bb98aadc0cd0fe44af3 on tablet 53765d0ffcc24bb98aadc0cd0fe44af3 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:51.137166 16895 raft_consensus.cc:482] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:51.137264 16895 raft_consensus.cc:504] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.137694 16895 leader_election.cc:288] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:51.138056 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017fe28f5f241af8955642664afaaa8" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:51.138211 9547 raft_consensus.cc:2454] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:51.138423 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017fe28f5f241af8955642664afaaa8" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:51.138536 7142 raft_consensus.cc:2454] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:51.138826 9350 leader_election.cc:302] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:51.139516 16895 raft_consensus.cc:2781] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:51.139580 16895 raft_consensus.cc:482] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:51.139613 16895 raft_consensus.cc:3037] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.140627 16895 raft_consensus.cc:504] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.140928 16895 leader_election.cc:288] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:51.141505 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017fe28f5f241af8955642664afaaa8" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:51.141598 7142 raft_consensus.cc:3037] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.142624 7142 raft_consensus.cc:2454] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:51.142899 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017fe28f5f241af8955642664afaaa8" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:51.143011 9547 raft_consensus.cc:3037] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.143966 9547 raft_consensus.cc:2454] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:51.144248 9350 leader_election.cc:302] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:51.144964 16895 raft_consensus.cc:2781] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:51.145044 16895 raft_consensus.cc:686] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:51.145161 16895 consensus_queue.cc:237] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:51.147642 6740 catalog_manager.cc:5484] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:51.163494 16672 raft_consensus.cc:482] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:51.163622 16672 raft_consensus.cc:504] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:51.164073 16672 leader_election.cc:288] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:51.164608 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5092f1e433f4f1cb27a169333aaecda" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:51.164750 7275 raft_consensus.cc:2454] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:51.164908 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5092f1e433f4f1cb27a169333aaecda" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:51.165055 7013 raft_consensus.cc:2454] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:51.165153 7079 leader_election.cc:302] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:51.165331 16672 raft_consensus.cc:2781] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:51.165375 16672 raft_consensus.cc:482] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:51.165417 16672 raft_consensus.cc:3037] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.166478 16672 raft_consensus.cc:504] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:51.167757 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5092f1e433f4f1cb27a169333aaecda" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:51.167893 7275 raft_consensus.cc:3037] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.168007 16672 leader_election.cc:288] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:51.168956 7275 raft_consensus.cc:2454] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:51.169212 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5092f1e433f4f1cb27a169333aaecda" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:51.169543 7079 leader_election.cc:302] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:51.169334 7013 raft_consensus.cc:3037] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.170493 7013 raft_consensus.cc:2454] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:51.171208 16672 raft_consensus.cc:2781] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:51.171298 16672 raft_consensus.cc:686] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:51.171433 16672 consensus_queue.cc:237] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:51.174439 6734 catalog_manager.cc:5484] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:51.235100 16548 raft_consensus.cc:482] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:51.235229 16548 raft_consensus.cc:504] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.235715 16548 leader_election.cc:288] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:51.236002 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d5f900639e2494b97e13f3703536f42" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:51.236048 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d5f900639e2494b97e13f3703536f42" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:51.236133 9285 raft_consensus.cc:2454] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:51.236167 9416 raft_consensus.cc:2454] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:51.236591 9483 leader_election.cc:302] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:51.237057 16548 raft_consensus.cc:2781] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:51.237123 16548 raft_consensus.cc:482] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:51.237174 16548 raft_consensus.cc:3037] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.238308 16548 raft_consensus.cc:504] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.238641 16548 leader_election.cc:288] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:51.238916 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d5f900639e2494b97e13f3703536f42" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:51.238946 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d5f900639e2494b97e13f3703536f42" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:51.239009 9285 raft_consensus.cc:3037] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.239073 9416 raft_consensus.cc:3037] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.239850 9285 raft_consensus.cc:2454] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:51.239923 9416 raft_consensus.cc:2454] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:51.240216 9482 leader_election.cc:302] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:51.240407 16548 raft_consensus.cc:2781] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:51.240586 16548 raft_consensus.cc:686] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:51.240916 16548 consensus_queue.cc:237] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.243098 6738 catalog_manager.cc:5484] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:51.244545 16672 raft_consensus.cc:482] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:51.244657 16672 raft_consensus.cc:504] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.245126 16672 leader_election.cc:288] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:51.245427 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d720846739e49f888da92befe06a309" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:51.245493 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d720846739e49f888da92befe06a309" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:51.245548 9416 raft_consensus.cc:2454] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:51.245615 6882 raft_consensus.cc:2454] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:51.245785 7077 leader_election.cc:302] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:51.245939 16672 raft_consensus.cc:2781] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:51.245988 16672 raft_consensus.cc:482] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:51.246021 16672 raft_consensus.cc:3037] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.246932 16672 raft_consensus.cc:504] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.247283 16672 leader_election.cc:288] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:51.247565 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d720846739e49f888da92befe06a309" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:51.247599 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d720846739e49f888da92befe06a309" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:51.247684 9417 raft_consensus.cc:3037] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.247700 6882 raft_consensus.cc:3037] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.248584 9417 raft_consensus.cc:2454] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:51.248584 6882 raft_consensus.cc:2454] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:51.248836 7079 leader_election.cc:302] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:51.249020 16672 raft_consensus.cc:2781] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:51.249120 16672 raft_consensus.cc:686] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:51.249258 16672 consensus_queue.cc:237] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.251188 6738 catalog_manager.cc:5484] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:51.271484 9397 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:51.275470 7122 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:51.276342 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:51.277900 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:51.284323 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:51.287334 9529 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:51.295934 6862 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:51.296190 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:51.298578 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:51.312338 7275 raft_consensus.cc:1261] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:51.312338 7013 raft_consensus.cc:1261] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:51.312726 16576 consensus_queue.cc:1035] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:51.313115 16672 consensus_queue.cc:1035] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:51.315975 6738 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet c5092f1e433f4f1cb27a169333aaecda (table rebalancer_dynamic_table [id=28697c769c2d411798cc40202e6c3862]). Expected version 1 got 0
I20240426 09:49:51.327819 6738 catalog_manager.cc:5576] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135) does not have the latest schema for tablet c5092f1e433f4f1cb27a169333aaecda (table rebalancer_dynamic_table [id=28697c769c2d411798cc40202e6c3862]). Expected version 1 got 0
I20240426 09:49:51.329727 6731 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet c5092f1e433f4f1cb27a169333aaecda (table rebalancer_dynamic_table [id=28697c769c2d411798cc40202e6c3862]). Expected version 1 got 0
I20240426 09:49:51.331934 16995 tablet.cc:1722] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:51.332311 16995 tablet.cc:1620] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:51.336797 6731 catalog_manager.cc:5576] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135) does not have the latest schema for tablet c5092f1e433f4f1cb27a169333aaecda (table rebalancer_dynamic_table [id=28697c769c2d411798cc40202e6c3862]). Expected version 1 got 0
I20240426 09:49:51.341720 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=28697c769c2d411798cc40202e6c3862] alter complete (version 1)
I20240426 09:49:51.344960 16997 tablet.cc:1722] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:51.345242 16997 tablet.cc:1620] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:51.349220 16996 tablet.cc:1722] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:51.350821 16996 tablet.cc:1620] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:51.382941 7142 consensus_queue.cc:237] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:51.384270 6882 raft_consensus.cc:1261] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:51.384327 9547 raft_consensus.cc:1261] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:51.384687 16672 consensus_queue.cc:1035] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:51.384860 16576 consensus_queue.cc:1035] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:51.386402 16672 raft_consensus.cc:2932] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:51.386854 9547 raft_consensus.cc:2932] T 956f89ba28c54473964cc877c26580f0 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:51.389485 6738 catalog_manager.cc:5484] T 956f89ba28c54473964cc877c26580f0 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:51.392649 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet 956f89ba28c54473964cc877c26580f0 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:51.392872 16893 tablet_replica.cc:330] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:51.393046 16893 raft_consensus.cc:2229] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:51.393191 16893 raft_consensus.cc:2258] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:51.393841 16893 ts_tablet_manager.cc:1875] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:51.395766 16893 ts_tablet_manager.cc:1888] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:51.395911 16893 log.cc:1196] T 956f89ba28c54473964cc877c26580f0 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/956f89ba28c54473964cc877c26580f0
I20240426 09:49:51.396430 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 956f89ba28c54473964cc877c26580f0 (table rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]) successfully deleted
I20240426 09:49:51.400295 6882 raft_consensus.cc:2932] T 956f89ba28c54473964cc877c26580f0 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:51.448091 6731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:51.453590 16995 tablet.cc:1722] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:51.453959 16995 tablet.cc:1620] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:51.454180 16996 tablet.cc:1722] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:51.454493 16996 tablet.cc:1620] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:51.459321 16997 tablet.cc:1722] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:51.459597 16997 tablet.cc:1620] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:51.465173 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=28697c769c2d411798cc40202e6c3862] alter complete (version 2)
I20240426 09:49:51.532778 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:51.537539 7122 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:51.548089 7142 raft_consensus.cc:1203] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:51.552098 9397 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:51.552531 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:51.559078 6862 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:51.568595 7253 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:51.569394 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:51.575978 15950 consensus_queue.cc:1035] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:51.580379 9529 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:51.581041 16995 tablet.cc:1722] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:51.586750 16997 tablet.cc:1722] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:51.593586 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:51.596483 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=28697c769c2d411798cc40202e6c3862] alter complete (version 3)
I20240426 09:49:51.601040 16996 tablet.cc:1722] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:51.609889 16895 consensus_queue.cc:1035] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:51.646474 16978 consensus_queue.cc:1035] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:51.650579 16548 consensus_queue.cc:1035] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:51.703437 6731 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:51.703699 6731 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:51.706315 6731 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet c5092f1e433f4f1cb27a169333aaecda
I20240426 09:49:51.706981 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet c5092f1e433f4f1cb27a169333aaecda with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:51 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:51.707386 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet c5092f1e433f4f1cb27a169333aaecda with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:51 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:51.708071 6993 tablet_service.cc:1514] Processing DeleteTablet for tablet c5092f1e433f4f1cb27a169333aaecda with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:51 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:51.708274 16893 tablet_replica.cc:330] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:51.708415 16893 raft_consensus.cc:2229] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:51.708524 16893 raft_consensus.cc:2258] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:51.709300 16893 ts_tablet_manager.cc:1875] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:51.711421 17127 tablet_replica.cc:330] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:51.711578 17127 raft_consensus.cc:2229] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:51.711607 16893 ts_tablet_manager.cc:1888] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:51.711678 16893 log.cc:1196] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/c5092f1e433f4f1cb27a169333aaecda
I20240426 09:49:51.711696 17127 raft_consensus.cc:2258] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:51.711949 16893 ts_tablet_manager.cc:1909] T c5092f1e433f4f1cb27a169333aaecda P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:51.715231 17127 ts_tablet_manager.cc:1875] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:51.717501 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet c5092f1e433f4f1cb27a169333aaecda (table rebalancer_dynamic_table_new_name [id=28697c769c2d411798cc40202e6c3862]) successfully deleted
I20240426 09:49:51.718569 17127 ts_tablet_manager.cc:1888] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:51.718691 17127 log.cc:1196] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/c5092f1e433f4f1cb27a169333aaecda
I20240426 09:49:51.719076 17127 ts_tablet_manager.cc:1909] T c5092f1e433f4f1cb27a169333aaecda P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:51.719801 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet c5092f1e433f4f1cb27a169333aaecda (table rebalancer_dynamic_table_new_name [id=28697c769c2d411798cc40202e6c3862]) successfully deleted
I20240426 09:49:51.721192 17128 tablet_replica.cc:330] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:51.721390 17128 raft_consensus.cc:2229] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:51.721618 17128 raft_consensus.cc:2258] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:51.722168 17128 ts_tablet_manager.cc:1875] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:51.724362 17128 ts_tablet_manager.cc:1888] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:51.724561 17128 log.cc:1196] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/c5092f1e433f4f1cb27a169333aaecda
I20240426 09:49:51.725478 17128 ts_tablet_manager.cc:1909] T c5092f1e433f4f1cb27a169333aaecda P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:51.727202 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet c5092f1e433f4f1cb27a169333aaecda (table rebalancer_dynamic_table_new_name [id=28697c769c2d411798cc40202e6c3862]) successfully deleted
I20240426 09:49:51.759107 7122 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:51.759282 9397 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:51.761871 16576 consensus_queue.cc:1035] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:51.769114 16576 consensus_queue.cc:1035] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:51.769776 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:51.771252 9266 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240426 09:49:51.774559 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:51.777571 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 6c2a1fd5edcf4323b61d7d8a4ea610f2 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.777849 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c2a1fd5edcf4323b61d7d8a4ea610f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.778127 7253 tablet_service.cc:1467] Processing CreateTablet for tablet d30be76596534071a3ea9a90375099f6 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.778390 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30be76596534071a3ea9a90375099f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.779999 9266 tablet_service.cc:1467] Processing CreateTablet for tablet d30be76596534071a3ea9a90375099f6 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.780241 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30be76596534071a3ea9a90375099f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.780292 7255 tablet_service.cc:1467] Processing CreateTablet for tablet efa472b0759c4cbbb7db13dc3fbe0b50 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.780496 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa472b0759c4cbbb7db13dc3fbe0b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.780748 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 6c2a1fd5edcf4323b61d7d8a4ea610f2 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.780966 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c2a1fd5edcf4323b61d7d8a4ea610f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.782645 9264 tablet_service.cc:1467] Processing CreateTablet for tablet efa472b0759c4cbbb7db13dc3fbe0b50 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.784696 6993 tablet_service.cc:1467] Processing CreateTablet for tablet efa472b0759c4cbbb7db13dc3fbe0b50 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.784945 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa472b0759c4cbbb7db13dc3fbe0b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.785987 6860 tablet_service.cc:1467] Processing CreateTablet for tablet 6c2a1fd5edcf4323b61d7d8a4ea610f2 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.786263 6860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c2a1fd5edcf4323b61d7d8a4ea610f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.787007 6862 tablet_service.cc:1467] Processing CreateTablet for tablet d30be76596534071a3ea9a90375099f6 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:51.787127 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa472b0759c4cbbb7db13dc3fbe0b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.787238 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30be76596534071a3ea9a90375099f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.790236 6862 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:51.792126 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:51.792393 9529 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:51.793952 17143 tablet_bootstrap.cc:492] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:51.795173 17146 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:51.801713 7255 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:51.803910 17144 tablet_bootstrap.cc:492] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:51.805011 17144 tablet_bootstrap.cc:654] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.805714 17145 tablet_bootstrap.cc:492] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:51.806723 17145 tablet_bootstrap.cc:654] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.813891 17145 tablet_bootstrap.cc:492] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:51.814050 17145 ts_tablet_manager.cc:1367] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240426 09:49:51.814508 17146 tablet_bootstrap.cc:654] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.814674 17145 raft_consensus.cc:348] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.814831 17145 raft_consensus.cc:374] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.814896 17145 raft_consensus.cc:729] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.815030 17145 consensus_queue.cc:260] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.821485 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:51.826930 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 97fc9bff511a45ac8c710c850c70857a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=38bc6ff9f1ac41d68a800905de89dde4]), partition=
I20240426 09:49:51.826928 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 97fc9bff511a45ac8c710c850c70857a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=38bc6ff9f1ac41d68a800905de89dde4]), partition=
I20240426 09:49:51.827194 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fc9bff511a45ac8c710c850c70857a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.827198 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fc9bff511a45ac8c710c850c70857a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.829594 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 97fc9bff511a45ac8c710c850c70857a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=38bc6ff9f1ac41d68a800905de89dde4]), partition=
I20240426 09:49:51.829826 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fc9bff511a45ac8c710c850c70857a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:51.834888 17143 tablet_bootstrap.cc:654] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.837200 17145 ts_tablet_manager.cc:1398] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.023s user 0.001s sys 0.000s
I20240426 09:49:51.837407 17145 tablet_bootstrap.cc:492] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:51.838438 17145 tablet_bootstrap.cc:654] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.840791 17146 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:51.840950 17146 ts_tablet_manager.cc:1367] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
I20240426 09:49:51.841596 17146 raft_consensus.cc:348] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:51.841775 17146 raft_consensus.cc:374] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.841853 17146 raft_consensus.cc:729] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.842669 17146 consensus_queue.cc:260] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:51.843132 17146 ts_tablet_manager.cc:1398] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:51.849829 17143 tablet_bootstrap.cc:492] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:51.849929 17143 ts_tablet_manager.cc:1367] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.056s user 0.001s sys 0.001s
I20240426 09:49:51.850351 17143 raft_consensus.cc:348] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.850602 17143 raft_consensus.cc:374] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.850646 17143 raft_consensus.cc:729] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.850797 17143 consensus_queue.cc:260] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.849566 17150 tablet_bootstrap.cc:492] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:51.851150 17143 ts_tablet_manager.cc:1398] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240426 09:49:51.851907 17150 tablet_bootstrap.cc:654] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.852600 17147 tablet_bootstrap.cc:492] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:51.853602 17147 tablet_bootstrap.cc:654] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.855077 17144 tablet_bootstrap.cc:492] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:51.855695 17144 ts_tablet_manager.cc:1367] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.000s
I20240426 09:49:51.855803 17150 tablet_bootstrap.cc:492] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:51.855927 17150 ts_tablet_manager.cc:1367] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:51.856245 17144 raft_consensus.cc:348] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.856386 17150 raft_consensus.cc:348] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.856523 17150 raft_consensus.cc:374] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.856582 17150 raft_consensus.cc:729] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.856737 17150 consensus_queue.cc:260] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.857028 17150 ts_tablet_manager.cc:1398] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.857523 17144 raft_consensus.cc:374] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.857620 17144 raft_consensus.cc:729] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.857786 17144 consensus_queue.cc:260] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.858009 17147 tablet_bootstrap.cc:492] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:51.858136 17147 ts_tablet_manager.cc:1367] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:51.858636 17144 ts_tablet_manager.cc:1398] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:51.859542 17144 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:51.860700 17144 tablet_bootstrap.cc:654] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.862313 17147 raft_consensus.cc:348] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.862449 17147 raft_consensus.cc:374] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.862519 17147 raft_consensus.cc:729] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.862654 17147 consensus_queue.cc:260] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.863384 17147 ts_tablet_manager.cc:1398] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:49:51.863689 17147 tablet_bootstrap.cc:492] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:51.864634 17145 tablet_bootstrap.cc:492] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:51.864722 17145 ts_tablet_manager.cc:1367] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240426 09:49:51.864730 17147 tablet_bootstrap.cc:654] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.865154 17145 raft_consensus.cc:348] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.865273 17145 raft_consensus.cc:374] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.865304 17145 raft_consensus.cc:729] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.865432 17145 consensus_queue.cc:260] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.865708 17145 ts_tablet_manager.cc:1398] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.865778 17147 tablet_bootstrap.cc:492] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:51.865849 17147 ts_tablet_manager.cc:1367] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:51.866221 17147 raft_consensus.cc:348] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.866323 17147 raft_consensus.cc:374] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.866361 17147 raft_consensus.cc:729] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.866475 17147 consensus_queue.cc:260] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.866750 17147 ts_tablet_manager.cc:1398] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.866910 17147 tablet_bootstrap.cc:492] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:51.867671 17145 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:51.867708 17147 tablet_bootstrap.cc:654] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.868703 17145 tablet_bootstrap.cc:654] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.869210 17147 tablet_bootstrap.cc:492] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:51.869282 17147 ts_tablet_manager.cc:1367] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:51.869694 17147 raft_consensus.cc:348] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.869796 17147 raft_consensus.cc:374] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.869832 17147 raft_consensus.cc:729] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.869959 17147 consensus_queue.cc:260] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.870205 17147 ts_tablet_manager.cc:1398] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:51.871228 17144 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:51.871346 17144 ts_tablet_manager.cc:1367] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:49:51.871781 17144 raft_consensus.cc:348] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:51.871886 17144 raft_consensus.cc:374] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.871915 17144 raft_consensus.cc:729] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.872018 17144 consensus_queue.cc:260] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:51.872355 17144 ts_tablet_manager.cc:1398] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.872519 17144 tablet_bootstrap.cc:492] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:51.873354 17144 tablet_bootstrap.cc:654] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:51.874058 17145 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:51.874122 17145 ts_tablet_manager.cc:1367] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:51.875797 17144 tablet_bootstrap.cc:492] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:51.876601 17145 raft_consensus.cc:348] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:51.876696 17145 raft_consensus.cc:374] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.876727 17145 raft_consensus.cc:729] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.876864 17145 consensus_queue.cc:260] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:51.877219 17145 ts_tablet_manager.cc:1398] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.877566 17144 ts_tablet_manager.cc:1367] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:51.878033 17144 raft_consensus.cc:348] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.878535 17144 raft_consensus.cc:374] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:51.878616 17144 raft_consensus.cc:729] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:51.878785 17144 consensus_queue.cc:260] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.879105 17144 ts_tablet_manager.cc:1398] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:51.971225 15178 raft_consensus.cc:482] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:51.971356 15178 raft_consensus.cc:504] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.971800 15178 leader_election.cc:288] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:51.972193 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fc9bff511a45ac8c710c850c70857a" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:51.972360 7275 raft_consensus.cc:2454] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:51.972873 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fc9bff511a45ac8c710c850c70857a" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:51.973014 9416 raft_consensus.cc:2454] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:51.973496 6815 leader_election.cc:302] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:51.973804 15178 raft_consensus.cc:2781] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:51.973855 15178 raft_consensus.cc:482] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:51.973891 15178 raft_consensus.cc:3037] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.975169 15178 raft_consensus.cc:504] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.975549 15178 leader_election.cc:288] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:51.976568 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fc9bff511a45ac8c710c850c70857a" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:51.976687 9417 raft_consensus.cc:3037] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.977656 9417 raft_consensus.cc:2454] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:51.977689 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fc9bff511a45ac8c710c850c70857a" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:51.977810 7275 raft_consensus.cc:3037] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:51.978870 7275 raft_consensus.cc:2454] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:51.979375 6815 leader_election.cc:302] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:51.979609 15178 raft_consensus.cc:2781] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:51.979712 15178 raft_consensus.cc:686] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:51.979857 15178 consensus_queue.cc:237] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:51.983901 6732 catalog_manager.cc:5484] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:52.032896 17072 raft_consensus.cc:482] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:52.033035 17072 raft_consensus.cc:504] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.033494 17072 leader_election.cc:288] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:52.033836 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa472b0759c4cbbb7db13dc3fbe0b50" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:52.033957 7275 raft_consensus.cc:2454] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:52.034199 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa472b0759c4cbbb7db13dc3fbe0b50" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:52.034240 9221 leader_election.cc:302] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:52.034343 7013 raft_consensus.cc:2454] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:52.034420 17072 raft_consensus.cc:2781] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:52.034461 17072 raft_consensus.cc:482] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:52.034493 17072 raft_consensus.cc:3037] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.035574 17072 raft_consensus.cc:504] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.035912 17072 leader_election.cc:288] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:52.036161 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa472b0759c4cbbb7db13dc3fbe0b50" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:52.036264 7013 raft_consensus.cc:3037] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.036232 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa472b0759c4cbbb7db13dc3fbe0b50" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:52.036332 7275 raft_consensus.cc:3037] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.037236 7013 raft_consensus.cc:2454] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:52.037247 7275 raft_consensus.cc:2454] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:52.037606 9219 leader_election.cc:302] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:52.037803 17072 raft_consensus.cc:2781] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:52.037891 17072 raft_consensus.cc:686] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:52.038053 17072 consensus_queue.cc:237] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.040014 6735 catalog_manager.cc:5484] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:52.092968 17072 raft_consensus.cc:482] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:52.093084 17072 raft_consensus.cc:504] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:52.093580 17072 leader_election.cc:288] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:52.093909 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2a1fd5edcf4323b61d7d8a4ea610f2" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:52.093962 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2a1fd5edcf4323b61d7d8a4ea610f2" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:52.094070 9672 raft_consensus.cc:2454] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:52.094120 6882 raft_consensus.cc:2454] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:52.094380 9221 leader_election.cc:302] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:52.094630 17072 raft_consensus.cc:2781] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:52.094707 17072 raft_consensus.cc:482] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:52.094784 17072 raft_consensus.cc:3037] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.095964 17072 raft_consensus.cc:504] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:52.096451 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:52.096670 17072 leader_election.cc:288] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:52.096704 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2a1fd5edcf4323b61d7d8a4ea610f2" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:52.096797 9672 raft_consensus.cc:3037] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.096786 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2a1fd5edcf4323b61d7d8a4ea610f2" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:52.096889 6882 raft_consensus.cc:3037] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.097641 9672 raft_consensus.cc:2454] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:52.097847 6882 raft_consensus.cc:2454] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:52.098093 9221 leader_election.cc:302] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:52.098244 17072 raft_consensus.cc:2781] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:52.098327 17072 raft_consensus.cc:686] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:52.098448 17072 consensus_queue.cc:237] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:52.101116 6731 catalog_manager.cc:5484] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:52.102635 9417 raft_consensus.cc:1261] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:52.103006 7274 raft_consensus.cc:1261] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:52.103322 15178 consensus_queue.cc:1035] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.103500 16574 consensus_queue.cc:1035] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.106374 17215 tablet.cc:1722] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:52.106520 6731 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 97fc9bff511a45ac8c710c850c70857a (table rebalancer_dynamic_table [id=38bc6ff9f1ac41d68a800905de89dde4]). Expected version 1 got 0
I20240426 09:49:52.106724 17215 tablet.cc:1620] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:52.106706 17214 tablet.cc:1722] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:52.107040 17214 tablet.cc:1620] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:52.108911 6735 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 97fc9bff511a45ac8c710c850c70857a (table rebalancer_dynamic_table [id=38bc6ff9f1ac41d68a800905de89dde4]). Expected version 1 got 0
I20240426 09:49:52.110508 6735 catalog_manager.cc:5576] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055) does not have the latest schema for tablet 97fc9bff511a45ac8c710c850c70857a (table rebalancer_dynamic_table [id=38bc6ff9f1ac41d68a800905de89dde4]). Expected version 1 got 0
I20240426 09:49:52.113512 17216 tablet.cc:1722] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:52.113793 17216 tablet.cc:1620] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:52.114328 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=38bc6ff9f1ac41d68a800905de89dde4] alter complete (version 1)
I20240426 09:49:52.122447 17154 raft_consensus.cc:482] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:52.122545 17154 raft_consensus.cc:504] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:52.123008 17154 leader_election.cc:288] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:52.123348 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30be76596534071a3ea9a90375099f6" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:52.123325 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30be76596534071a3ea9a90375099f6" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:52.123471 6882 raft_consensus.cc:2454] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:52.123494 9285 raft_consensus.cc:2454] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:52.123719 7209 leader_election.cc:302] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:52.123883 17154 raft_consensus.cc:2781] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:52.123941 17154 raft_consensus.cc:482] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:52.123989 17154 raft_consensus.cc:3037] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.124986 17154 raft_consensus.cc:504] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:52.125280 17154 leader_election.cc:288] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:52.125545 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30be76596534071a3ea9a90375099f6" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:52.125552 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30be76596534071a3ea9a90375099f6" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:52.125654 6882 raft_consensus.cc:3037] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.125654 9285 raft_consensus.cc:3037] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.126583 9285 raft_consensus.cc:2454] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:52.126583 6882 raft_consensus.cc:2454] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:52.126863 7210 leader_election.cc:302] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:52.127123 17154 raft_consensus.cc:2781] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:52.127223 17154 raft_consensus.cc:686] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:52.127357 17154 consensus_queue.cc:237] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:52.129194 6734 catalog_manager.cc:5484] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:52.163975 9265 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:52.165953 9397 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:52.172500 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:52.177287 9529 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:52.187393 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:52.191793 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:52.198303 7122 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:52.216174 6734 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:52.221483 17214 tablet.cc:1722] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:52.221837 17214 tablet.cc:1620] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:52.221891 17216 tablet.cc:1722] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:52.221997 17215 tablet.cc:1722] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:52.222164 17216 tablet.cc:1620] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:52.222262 17215 tablet.cc:1620] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:52.227492 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=38bc6ff9f1ac41d68a800905de89dde4] alter complete (version 2)
I20240426 09:49:52.242095 7255 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:52.271230 9672 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "53765d0ffcc24bb98aadc0cd0fe44af3"
dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39206
I20240426 09:49:52.271401 9672 raft_consensus.cc:595] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:52.333248 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:52.339107 17214 tablet.cc:1722] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:52.340178 17216 tablet.cc:1722] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:52.342273 17215 tablet.cc:1722] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:52.346208 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=38bc6ff9f1ac41d68a800905de89dde4] alter complete (version 3)
I20240426 09:49:52.390970 7122 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:52.393721 9397 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:52.397285 9529 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:52.401093 9661 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:52.405609 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:52.407789 9265 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:52.422495 7255 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:52.438290 17072 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.447340 16791 raft_consensus.cc:979] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: : Instructing follower c244e02f3cf847c0a5b69e1059f1071c to start an election
I20240426 09:49:52.447427 16791 raft_consensus.cc:1067] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Signalling peer c244e02f3cf847c0a5b69e1059f1071c to start an election
I20240426 09:49:52.447554 16894 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.448184 6880 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "53765d0ffcc24bb98aadc0cd0fe44af3"
dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
from {username='slave'} at 127.0.99.136:56221
I20240426 09:49:52.448315 6880 raft_consensus.cc:482] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:52.448381 6880 raft_consensus.cc:3037] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:52.449416 6880 raft_consensus.cc:504] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.452121 6732 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:52.452250 6732 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:52.452638 6880 leader_election.cc:288] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 2 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:52.453315 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53765d0ffcc24bb98aadc0cd0fe44af3" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:52.453438 9547 raft_consensus.cc:3037] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:52.454553 9547 raft_consensus.cc:2454] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 2.
I20240426 09:49:52.454605 6732 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 97fc9bff511a45ac8c710c850c70857a
I20240426 09:49:52.455247 6993 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:52.455235 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53765d0ffcc24bb98aadc0cd0fe44af3" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:52.455358 9672 raft_consensus.cc:3032] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:52.455401 9672 raft_consensus.cc:729] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:52.455601 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 97fc9bff511a45ac8c710c850c70857a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:52 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:52.455572 9672 consensus_queue.cc:260] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.455775 9672 raft_consensus.cc:3037] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:52.456696 9672 raft_consensus.cc:2454] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 2.
I20240426 09:49:52.456981 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet 97fc9bff511a45ac8c710c850c70857a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:52 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:52.457154 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 97fc9bff511a45ac8c710c850c70857a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:52 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:52.457504 6815 leader_election.cc:302] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:52.457728 15178 raft_consensus.cc:2781] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:52.457794 15178 raft_consensus.cc:686] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:52.457902 15178 consensus_queue.cc:237] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.459805 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53765d0ffcc24bb98aadc0cd0fe44af3" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:52.459936 7142 raft_consensus.cc:3037] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:52.460369 6732 catalog_manager.cc:5484] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 1 to 2, leader changed from 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 2 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:52.460937 7142 raft_consensus.cc:2454] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 2.
I20240426 09:49:52.465407 17288 tablet_replica.cc:330] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:52.465853 17288 raft_consensus.cc:2229] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:52.466043 17288 raft_consensus.cc:2258] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:52.466296 17286 tablet_replica.cc:330] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:52.466468 17286 raft_consensus.cc:2229] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:52.466598 17288 ts_tablet_manager.cc:1875] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:52.466630 17286 raft_consensus.cc:2258] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:52.466879 17287 tablet_replica.cc:330] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:52.467026 17287 raft_consensus.cc:2229] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:52.467442 17286 ts_tablet_manager.cc:1875] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:52.468930 17288 ts_tablet_manager.cc:1888] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:52.469044 17288 log.cc:1196] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/97fc9bff511a45ac8c710c850c70857a
I20240426 09:49:52.469360 17288 ts_tablet_manager.cc:1909] T 97fc9bff511a45ac8c710c850c70857a P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:52.469974 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 97fc9bff511a45ac8c710c850c70857a (table rebalancer_dynamic_table_new_name [id=38bc6ff9f1ac41d68a800905de89dde4]) successfully deleted
I20240426 09:49:52.470973 17287 raft_consensus.cc:2258] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:52.471540 17287 ts_tablet_manager.cc:1875] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:52.472340 17286 ts_tablet_manager.cc:1888] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:52.472410 17286 log.cc:1196] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/97fc9bff511a45ac8c710c850c70857a
I20240426 09:49:52.472965 17286 ts_tablet_manager.cc:1909] T 97fc9bff511a45ac8c710c850c70857a P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:52.473676 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 97fc9bff511a45ac8c710c850c70857a (table rebalancer_dynamic_table_new_name [id=38bc6ff9f1ac41d68a800905de89dde4]) successfully deleted
I20240426 09:49:52.474493 17287 ts_tablet_manager.cc:1888] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:52.474575 17287 log.cc:1196] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/97fc9bff511a45ac8c710c850c70857a
I20240426 09:49:52.474962 17287 ts_tablet_manager.cc:1909] T 97fc9bff511a45ac8c710c850c70857a P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:52.475535 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 97fc9bff511a45ac8c710c850c70857a (table rebalancer_dynamic_table_new_name [id=38bc6ff9f1ac41d68a800905de89dde4]) successfully deleted
I20240426 09:49:52.549945 16894 consensus_queue.cc:1035] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.553915 16894 consensus_queue.cc:1035] T 6c2a1fd5edcf4323b61d7d8a4ea610f2 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.556569 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:52.562935 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 987d367e17644b00adc93776c1914cf4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=497fb60cc7c1466791a1869b92ff8966]), partition=
I20240426 09:49:52.563208 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987d367e17644b00adc93776c1914cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.563200 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 987d367e17644b00adc93776c1914cf4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=497fb60cc7c1466791a1869b92ff8966]), partition=
I20240426 09:49:52.563407 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987d367e17644b00adc93776c1914cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.565588 17304 tablet_bootstrap.cc:492] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:52.566533 17304 tablet_bootstrap.cc:654] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.566802 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 987d367e17644b00adc93776c1914cf4 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=497fb60cc7c1466791a1869b92ff8966]), partition=
I20240426 09:49:52.566998 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 987d367e17644b00adc93776c1914cf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.567140 17303 tablet_bootstrap.cc:492] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:52.567970 17303 tablet_bootstrap.cc:654] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.568819 17304 tablet_bootstrap.cc:492] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:52.568890 17304 ts_tablet_manager.cc:1367] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:52.569155 17307 tablet_bootstrap.cc:492] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:52.569378 17304 raft_consensus.cc:348] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:52.569506 17304 raft_consensus.cc:374] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.569831 17304 raft_consensus.cc:729] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.570741 17307 tablet_bootstrap.cc:654] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.569960 17304 consensus_queue.cc:260] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:52.571609 17304 ts_tablet_manager.cc:1398] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:52.572116 17303 tablet_bootstrap.cc:492] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:52.572206 17303 ts_tablet_manager.cc:1367] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:52.572805 17303 raft_consensus.cc:348] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:52.573065 17303 raft_consensus.cc:374] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.573138 17303 raft_consensus.cc:729] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.573341 17303 consensus_queue.cc:260] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:52.576323 17303 ts_tablet_manager.cc:1398] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:52.583554 17307 tablet_bootstrap.cc:492] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:52.583663 17307 ts_tablet_manager.cc:1367] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.001s
I20240426 09:49:52.584156 17307 raft_consensus.cc:348] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:52.584270 17307 raft_consensus.cc:374] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.584302 17307 raft_consensus.cc:729] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.584465 17307 consensus_queue.cc:260] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:52.584784 17307 ts_tablet_manager.cc:1398] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:52.606850 16548 raft_consensus.cc:482] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:52.606966 16548 raft_consensus.cc:504] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:52.607359 16548 leader_election.cc:288] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:52.607729 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987d367e17644b00adc93776c1914cf4" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:52.607720 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987d367e17644b00adc93776c1914cf4" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:52.607862 7142 raft_consensus.cc:2454] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:52.607879 9416 raft_consensus.cc:2454] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:52.608124 9482 leader_election.cc:302] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:52.608261 16548 raft_consensus.cc:2781] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:52.608319 16548 raft_consensus.cc:482] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:52.608346 16548 raft_consensus.cc:3037] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.609195 16548 raft_consensus.cc:504] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:52.609530 16548 leader_election.cc:288] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:52.609874 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987d367e17644b00adc93776c1914cf4" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:52.609987 7142 raft_consensus.cc:3037] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.609989 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "987d367e17644b00adc93776c1914cf4" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:52.610081 9416 raft_consensus.cc:3037] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.611030 7142 raft_consensus.cc:2454] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:52.611030 9416 raft_consensus.cc:2454] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:52.611320 9482 leader_election.cc:302] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:52.611500 16548 raft_consensus.cc:2781] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:52.611588 16548 raft_consensus.cc:686] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:52.611747 16548 consensus_queue.cc:237] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:52.613974 6732 catalog_manager.cc:5484] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:52.672535 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:52.677692 17154 consensus_queue.cc:1035] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.679950 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 87d356a7ac32469c824af0970730214f (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:52.680090 6993 tablet_service.cc:1467] Processing CreateTablet for tablet b674ff3fd5284b02b7e5b3fc62145a0a (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:52.680270 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87d356a7ac32469c824af0970730214f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.680358 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b674ff3fd5284b02b7e5b3fc62145a0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.680298 9529 tablet_service.cc:1467] Processing CreateTablet for tablet b674ff3fd5284b02b7e5b3fc62145a0a (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:52.680548 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b674ff3fd5284b02b7e5b3fc62145a0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.682343 17312 tablet_bootstrap.cc:492] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:52.682525 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 87d356a7ac32469c824af0970730214f (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:52.682801 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87d356a7ac32469c824af0970730214f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.682803 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3fb0b19aa34a53831c27c4d7490007 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:52.683035 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3fb0b19aa34a53831c27c4d7490007. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.683732 17312 tablet_bootstrap.cc:654] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.684986 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3fb0b19aa34a53831c27c4d7490007 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:52.685235 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3fb0b19aa34a53831c27c4d7490007. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.685420 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 6e3fb0b19aa34a53831c27c4d7490007 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:52.685688 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e3fb0b19aa34a53831c27c4d7490007. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.685724 9397 tablet_service.cc:1467] Processing CreateTablet for tablet b674ff3fd5284b02b7e5b3fc62145a0a (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:52.685950 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b674ff3fd5284b02b7e5b3fc62145a0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.687350 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 87d356a7ac32469c824af0970730214f (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:52.687544 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87d356a7ac32469c824af0970730214f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:52.688056 17304 tablet_bootstrap.cc:492] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:52.682863 17303 tablet_bootstrap.cc:492] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:52.688975 17304 tablet_bootstrap.cc:654] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.689262 17303 tablet_bootstrap.cc:654] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.690289 17304 tablet_bootstrap.cc:492] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:52.690364 17304 ts_tablet_manager.cc:1367] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:52.691138 17304 raft_consensus.cc:348] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.691247 17304 raft_consensus.cc:374] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.691279 17304 raft_consensus.cc:729] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.691394 17304 consensus_queue.cc:260] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.691709 17304 ts_tablet_manager.cc:1398] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:52.692709 17303 tablet_bootstrap.cc:492] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:52.692798 17303 ts_tablet_manager.cc:1367] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:52.693109 17307 tablet_bootstrap.cc:492] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:52.693265 17303 raft_consensus.cc:348] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:52.693373 17303 raft_consensus.cc:374] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.693405 17303 raft_consensus.cc:729] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.693517 17303 consensus_queue.cc:260] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:52.693836 17303 ts_tablet_manager.cc:1398] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:52.696314 16735 consensus_queue.cc:1035] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.696818 17307 tablet_bootstrap.cc:654] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.697633 17315 tablet_bootstrap.cc:492] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:52.698278 17313 tablet_bootstrap.cc:492] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:52.698510 17316 tablet_bootstrap.cc:492] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:52.698619 17315 tablet_bootstrap.cc:654] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.699822 17312 tablet_bootstrap.cc:492] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:52.699946 17312 ts_tablet_manager.cc:1367] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240426 09:49:52.700448 17312 raft_consensus.cc:348] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.700557 17312 raft_consensus.cc:374] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.700590 17312 raft_consensus.cc:729] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.700721 17312 consensus_queue.cc:260] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.700994 17312 ts_tablet_manager.cc:1398] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:52.701844 17303 tablet_bootstrap.cc:492] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:52.701983 17313 tablet_bootstrap.cc:654] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.702239 17316 tablet_bootstrap.cc:654] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.702214 16672 raft_consensus.cc:482] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:52.702304 16672 raft_consensus.cc:504] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.702548 17315 tablet_bootstrap.cc:492] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:52.702625 17315 ts_tablet_manager.cc:1367] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:52.702687 16672 leader_election.cc:288] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:52.702930 17303 tablet_bootstrap.cc:654] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.703119 17315 raft_consensus.cc:348] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.703250 17315 raft_consensus.cc:374] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.703341 17315 raft_consensus.cc:729] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.703313 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3fb0b19aa34a53831c27c4d7490007" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:52.703517 17315 consensus_queue.cc:260] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.703712 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3fb0b19aa34a53831c27c4d7490007" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:52.703922 17315 ts_tablet_manager.cc:1398] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20240426 09:49:52.704170 7077 leader_election.cc:341] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Illegal state: must be running to vote when last-logged opid is not known
W20240426 09:49:52.704313 7077 leader_election.cc:341] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:52.704355 7077 leader_election.cc:302] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441; no voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:52.704578 17313 tablet_bootstrap.cc:492] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:52.704663 17313 ts_tablet_manager.cc:1367] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:52.704741 16672 raft_consensus.cc:2726] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240426 09:49:52.705243 17313 raft_consensus.cc:348] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:52.705420 17313 raft_consensus.cc:374] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.705579 17313 raft_consensus.cc:729] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.705732 17313 consensus_queue.cc:260] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:52.706126 17313 ts_tablet_manager.cc:1398] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:52.715642 17303 tablet_bootstrap.cc:492] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:52.715817 17303 ts_tablet_manager.cc:1367] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240426 09:49:52.716291 17303 raft_consensus.cc:348] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.716727 17303 raft_consensus.cc:374] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.716744 17307 tablet_bootstrap.cc:492] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:52.716804 17303 raft_consensus.cc:729] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.716831 17307 ts_tablet_manager.cc:1367] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.002s
I20240426 09:49:52.716967 17303 consensus_queue.cc:260] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.717279 17307 raft_consensus.cc:348] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:52.717379 17307 raft_consensus.cc:374] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.717411 17307 raft_consensus.cc:729] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.717536 17307 consensus_queue.cc:260] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:52.717839 17307 ts_tablet_manager.cc:1398] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:52.719144 17316 tablet_bootstrap.cc:492] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:52.719241 17316 ts_tablet_manager.cc:1367] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240426 09:49:52.719698 17316 raft_consensus.cc:348] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.719818 17316 raft_consensus.cc:374] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.719848 17316 raft_consensus.cc:729] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.719969 17316 consensus_queue.cc:260] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.720275 17316 ts_tablet_manager.cc:1398] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:52.720594 17316 tablet_bootstrap.cc:492] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:52.721516 17316 tablet_bootstrap.cc:654] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:52.722646 17316 tablet_bootstrap.cc:492] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:52.722716 17316 ts_tablet_manager.cc:1367] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:49:52.722684 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:52.723093 17316 raft_consensus.cc:348] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.723183 17316 raft_consensus.cc:374] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:52.723233 17316 raft_consensus.cc:729] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:52.723341 17316 consensus_queue.cc:260] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.723592 17316 ts_tablet_manager.cc:1398] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:52.728190 17303 ts_tablet_manager.cc:1398] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20240426 09:49:52.733382 9416 raft_consensus.cc:1261] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:52.733945 16548 consensus_queue.cc:1035] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.734192 7142 raft_consensus.cc:1261] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:52.736487 16548 consensus_queue.cc:1035] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:52.738092 17215 tablet.cc:1722] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:52.738399 17215 tablet.cc:1620] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:52.739291 6740 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet 987d367e17644b00adc93776c1914cf4 (table rebalancer_dynamic_table [id=497fb60cc7c1466791a1869b92ff8966]). Expected version 1 got 0
I20240426 09:49:52.744412 6732 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 987d367e17644b00adc93776c1914cf4 (table rebalancer_dynamic_table [id=497fb60cc7c1466791a1869b92ff8966]). Expected version 1 got 0
I20240426 09:49:52.747007 17339 tablet.cc:1722] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:52.747509 17339 tablet.cc:1620] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:52.749727 6740 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet 987d367e17644b00adc93776c1914cf4 (table rebalancer_dynamic_table [id=497fb60cc7c1466791a1869b92ff8966]). Expected version 1 got 0
I20240426 09:49:52.749882 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=497fb60cc7c1466791a1869b92ff8966] alter complete (version 1)
I20240426 09:49:52.756606 17342 tablet.cc:1722] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:52.759125 17342 tablet.cc:1620] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:52.814601 15237 raft_consensus.cc:482] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:52.814741 15237 raft_consensus.cc:504] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:52.815232 15237 leader_election.cc:288] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:52.815972 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b674ff3fd5284b02b7e5b3fc62145a0a" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:52.816107 9416 raft_consensus.cc:2454] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:52.816345 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b674ff3fd5284b02b7e5b3fc62145a0a" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:52.816473 9547 raft_consensus.cc:2454] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:52.816987 6946 leader_election.cc:302] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:52.817270 15237 raft_consensus.cc:2781] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:52.817327 15237 raft_consensus.cc:482] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:52.817381 15237 raft_consensus.cc:3037] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.818500 15237 raft_consensus.cc:504] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:52.818835 15237 leader_election.cc:288] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:52.819388 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b674ff3fd5284b02b7e5b3fc62145a0a" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:52.819466 9547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b674ff3fd5284b02b7e5b3fc62145a0a" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:52.819504 9416 raft_consensus.cc:3037] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.819558 9547 raft_consensus.cc:3037] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.820482 9416 raft_consensus.cc:2454] T b674ff3fd5284b02b7e5b3fc62145a0a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:52.820498 9547 raft_consensus.cc:2454] T b674ff3fd5284b02b7e5b3fc62145a0a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:52.820840 6946 leader_election.cc:302] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:52.821166 15237 raft_consensus.cc:2781] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:52.821251 15237 raft_consensus.cc:686] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:52.821383 15237 consensus_queue.cc:237] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:52.824070 6735 catalog_manager.cc:5484] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:52.834405 17072 raft_consensus.cc:482] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:52.834507 17072 raft_consensus.cc:504] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.834923 17072 leader_election.cc:288] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:52.835328 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d356a7ac32469c824af0970730214f" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:52.835469 7275 raft_consensus.cc:2454] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:52.835784 9221 leader_election.cc:302] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:52.835819 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d356a7ac32469c824af0970730214f" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:52.835947 9672 raft_consensus.cc:2454] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:52.836102 16894 raft_consensus.cc:2781] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:52.836165 16894 raft_consensus.cc:482] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:52.836212 16894 raft_consensus.cc:3037] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.837002 16894 raft_consensus.cc:504] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.837327 16894 leader_election.cc:288] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:52.837648 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d356a7ac32469c824af0970730214f" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:52.837755 7275 raft_consensus.cc:3037] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.837881 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d356a7ac32469c824af0970730214f" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:52.837989 9672 raft_consensus.cc:3037] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.838811 7275 raft_consensus.cc:2454] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:52.839071 9672 raft_consensus.cc:2454] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:52.839087 9221 leader_election.cc:302] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:52.839370 17072 raft_consensus.cc:2781] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:52.839452 17072 raft_consensus.cc:686] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:52.839584 17072 consensus_queue.cc:237] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:52.841897 6740 catalog_manager.cc:5484] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:52.847594 17309 raft_consensus.cc:482] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:52.847685 17309 raft_consensus.cc:504] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.848017 17309 leader_election.cc:288] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:52.848326 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3fb0b19aa34a53831c27c4d7490007" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:52.848454 9672 raft_consensus.cc:2454] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:52.848444 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3fb0b19aa34a53831c27c4d7490007" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:52.848562 7142 raft_consensus.cc:2454] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:52.848691 9482 leader_election.cc:302] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:52.848891 17309 raft_consensus.cc:2781] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:52.848939 17309 raft_consensus.cc:482] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:52.848968 17309 raft_consensus.cc:3037] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.849927 17309 raft_consensus.cc:504] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.850193 17309 leader_election.cc:288] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:52.850549 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3fb0b19aa34a53831c27c4d7490007" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:52.850564 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e3fb0b19aa34a53831c27c4d7490007" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:52.850666 9672 raft_consensus.cc:3037] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.850666 7142 raft_consensus.cc:3037] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:52.851639 9672 raft_consensus.cc:2454] T 6e3fb0b19aa34a53831c27c4d7490007 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:52.851639 7142 raft_consensus.cc:2454] T 6e3fb0b19aa34a53831c27c4d7490007 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:52.851941 9482 leader_election.cc:302] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:52.852097 17309 raft_consensus.cc:2781] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:52.852177 17309 raft_consensus.cc:686] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:52.852320 17309 consensus_queue.cc:237] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:52.855135 6740 catalog_manager.cc:5484] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:52.856580 6734 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:52.862141 17339 tablet.cc:1722] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:52.862496 17339 tablet.cc:1620] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:52.863202 17215 tablet.cc:1722] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:52.863492 17215 tablet.cc:1620] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:52.872761 17342 tablet.cc:1722] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:52.873076 17342 tablet.cc:1620] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:52.877121 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=497fb60cc7c1466791a1869b92ff8966] alter complete (version 2)
I20240426 09:49:52.878754 6993 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:52.882108 7122 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:52.889110 9265 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:52.891813 9397 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:52.895192 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:52.898027 9660 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:52.918946 9547 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:49:52.919525 16574 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:52.923611 9527 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:52.946499 7255 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:52.986361 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:52.991551 17339 tablet.cc:1722] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:52.991887 17215 tablet.cc:1722] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:52.992533 17342 tablet.cc:1722] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:52.997215 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=497fb60cc7c1466791a1869b92ff8966] alter complete (version 3)
I20240426 09:49:53.062251 9672 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:49:53.062709 16574 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:53.064970 7142 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:49:53.065601 17229 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:53.100066 6740 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:53.100253 6740 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:53.102375 6740 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 987d367e17644b00adc93776c1914cf4
I20240426 09:49:53.102948 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 987d367e17644b00adc93776c1914cf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:53 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:53.103101 7122 tablet_service.cc:1514] Processing DeleteTablet for tablet 987d367e17644b00adc93776c1914cf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:53 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:53.103279 17392 tablet_replica.cc:330] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:53.103433 17392 raft_consensus.cc:2229] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:53.103519 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 987d367e17644b00adc93776c1914cf4 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:53 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:53.103408 17393 tablet_replica.cc:330] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:53.103652 17392 raft_consensus.cc:2258] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:53.103799 17393 raft_consensus.cc:2229] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:53.104014 17393 raft_consensus.cc:2258] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:53.104207 17392 ts_tablet_manager.cc:1875] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:53.104568 17393 ts_tablet_manager.cc:1875] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:53.105368 17394 tablet_replica.cc:330] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:53.105535 17394 raft_consensus.cc:2229] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:53.105670 17394 raft_consensus.cc:2258] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:53.106089 17392 ts_tablet_manager.cc:1888] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:53.106189 17392 log.cc:1196] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/987d367e17644b00adc93776c1914cf4
I20240426 09:49:53.106240 17394 ts_tablet_manager.cc:1875] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:53.106802 17392 ts_tablet_manager.cc:1909] T 987d367e17644b00adc93776c1914cf4 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:53.107028 17393 ts_tablet_manager.cc:1888] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:53.107100 17393 log.cc:1196] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/987d367e17644b00adc93776c1914cf4
I20240426 09:49:53.107450 17393 ts_tablet_manager.cc:1909] T 987d367e17644b00adc93776c1914cf4 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:53.107486 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 987d367e17644b00adc93776c1914cf4 (table rebalancer_dynamic_table_new_name [id=497fb60cc7c1466791a1869b92ff8966]) successfully deleted
I20240426 09:49:53.108412 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 987d367e17644b00adc93776c1914cf4 (table rebalancer_dynamic_table_new_name [id=497fb60cc7c1466791a1869b92ff8966]) successfully deleted
I20240426 09:49:53.108469 17394 ts_tablet_manager.cc:1888] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:53.108551 17394 log.cc:1196] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/987d367e17644b00adc93776c1914cf4
I20240426 09:49:53.108773 17394 ts_tablet_manager.cc:1909] T 987d367e17644b00adc93776c1914cf4 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:53.109268 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 987d367e17644b00adc93776c1914cf4 (table rebalancer_dynamic_table_new_name [id=497fb60cc7c1466791a1869b92ff8966]) successfully deleted
I20240426 09:49:53.204025 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:53.211414 9397 tablet_service.cc:1467] Processing CreateTablet for tablet da96939413f24cebb13ff7744a6624f3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=67cb01835dac48cabfac4e93e0b7f4a0]), partition=
I20240426 09:49:53.211659 9660 tablet_service.cc:1467] Processing CreateTablet for tablet da96939413f24cebb13ff7744a6624f3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=67cb01835dac48cabfac4e93e0b7f4a0]), partition=
I20240426 09:49:53.211696 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da96939413f24cebb13ff7744a6624f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.211856 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da96939413f24cebb13ff7744a6624f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.213979 17307 tablet_bootstrap.cc:492] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:53.214176 7255 tablet_service.cc:1467] Processing CreateTablet for tablet da96939413f24cebb13ff7744a6624f3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=67cb01835dac48cabfac4e93e0b7f4a0]), partition=
I20240426 09:49:53.214438 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da96939413f24cebb13ff7744a6624f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.214947 17307 tablet_bootstrap.cc:654] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.215544 17316 tablet_bootstrap.cc:492] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:53.216393 17316 tablet_bootstrap.cc:654] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.216822 17418 tablet_bootstrap.cc:492] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:53.217449 17307 tablet_bootstrap.cc:492] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:53.217555 17307 ts_tablet_manager.cc:1367] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:53.217720 17316 tablet_bootstrap.cc:492] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:53.217813 17316 ts_tablet_manager.cc:1367] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:53.218101 17307 raft_consensus.cc:348] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:53.218199 17307 raft_consensus.cc:374] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.218230 17307 raft_consensus.cc:729] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.218284 17316 raft_consensus.cc:348] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:53.218401 17316 raft_consensus.cc:374] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.218379 17307 consensus_queue.cc:260] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:53.218433 17316 raft_consensus.cc:729] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.218549 17316 consensus_queue.cc:260] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:53.218856 17316 ts_tablet_manager.cc:1398] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:53.219516 17307 ts_tablet_manager.cc:1398] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:53.221562 17418 tablet_bootstrap.cc:654] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.222832 17418 tablet_bootstrap.cc:492] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:53.222970 17418 ts_tablet_manager.cc:1367] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:53.223678 17418 raft_consensus.cc:348] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:53.224887 17418 raft_consensus.cc:374] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.224970 17418 raft_consensus.cc:729] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.225143 17418 consensus_queue.cc:260] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:53.225766 17418 ts_tablet_manager.cc:1398] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:53.243119 16894 consensus_queue.cc:1035] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:53.247681 16894 consensus_queue.cc:1035] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:53.348681 17387 consensus_queue.cc:1035] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:53.351799 16548 consensus_queue.cc:1035] T 6e3fb0b19aa34a53831c27c4d7490007 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:53.359297 15237 consensus_queue.cc:1035] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:53.362048 16849 consensus_queue.cc:1035] T b674ff3fd5284b02b7e5b3fc62145a0a P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:53.400986 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:53.406909 6993 tablet_service.cc:1467] Processing CreateTablet for tablet ecd7063af4854e17bb2211e31033a89b (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:53.407269 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd7063af4854e17bb2211e31033a89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.407383 7122 tablet_service.cc:1467] Processing CreateTablet for tablet ecd7063af4854e17bb2211e31033a89b (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:53.407685 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd7063af4854e17bb2211e31033a89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.407702 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 348e90f5c63e4eda89b87fc103896302 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:53.407836 6992 tablet_service.cc:1467] Processing CreateTablet for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:53.407931 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348e90f5c63e4eda89b87fc103896302. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.408056 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.409790 9527 tablet_service.cc:1467] Processing CreateTablet for tablet ecd7063af4854e17bb2211e31033a89b (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:53.410043 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd7063af4854e17bb2211e31033a89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.410106 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 348e90f5c63e4eda89b87fc103896302 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:53.410220 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:53.410329 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348e90f5c63e4eda89b87fc103896302. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.410477 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.412212 17452 tablet_bootstrap.cc:492] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:53.413029 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 348e90f5c63e4eda89b87fc103896302 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:53.413262 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348e90f5c63e4eda89b87fc103896302. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.413945 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:53.414167 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:53.414222 17451 tablet_bootstrap.cc:492] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:53.414249 17452 tablet_bootstrap.cc:654] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.415620 17451 tablet_bootstrap.cc:654] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.416000 17453 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:53.416985 17450 tablet_bootstrap.cc:492] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:53.417125 17453 tablet_bootstrap.cc:654] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.417956 17450 tablet_bootstrap.cc:654] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.418339 17451 tablet_bootstrap.cc:492] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:53.418452 17451 ts_tablet_manager.cc:1367] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:53.419102 17452 tablet_bootstrap.cc:492] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:53.419222 17452 ts_tablet_manager.cc:1367] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:53.419320 17450 tablet_bootstrap.cc:492] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:53.419394 17450 ts_tablet_manager.cc:1367] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:53.419684 17452 raft_consensus.cc:348] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.419783 17452 raft_consensus.cc:374] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.419816 17452 raft_consensus.cc:729] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.419818 17453 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:53.419895 17453 ts_tablet_manager.cc:1367] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:53.419929 17452 consensus_queue.cc:260] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.420235 17452 ts_tablet_manager.cc:1398] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:53.419970 17450 raft_consensus.cc:348] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.420429 17452 tablet_bootstrap.cc:492] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:53.420413 17450 raft_consensus.cc:374] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.420380 17453 raft_consensus.cc:348] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:53.420460 17450 raft_consensus.cc:729] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.420483 17453 raft_consensus.cc:374] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.420521 17453 raft_consensus.cc:729] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.420598 17450 consensus_queue.cc:260] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.420635 17453 consensus_queue.cc:260] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:53.420883 17450 ts_tablet_manager.cc:1398] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:53.421180 17450 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:53.421308 17452 tablet_bootstrap.cc:654] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.420943 17453 ts_tablet_manager.cc:1398] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:53.422084 17450 tablet_bootstrap.cc:654] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.422293 17452 tablet_bootstrap.cc:492] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:53.422350 17452 ts_tablet_manager.cc:1367] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240426 09:49:53.422792 17452 raft_consensus.cc:348] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:53.422914 17452 raft_consensus.cc:374] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.422948 17452 raft_consensus.cc:729] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.423097 17452 consensus_queue.cc:260] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:53.423372 17452 ts_tablet_manager.cc:1398] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:53.424998 17450 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:53.425083 17450 ts_tablet_manager.cc:1367] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:53.425732 17450 raft_consensus.cc:348] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:53.425849 17450 raft_consensus.cc:374] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.425881 17450 raft_consensus.cc:729] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.425995 17450 consensus_queue.cc:260] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:53.426249 17450 ts_tablet_manager.cc:1398] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:53.426394 17450 tablet_bootstrap.cc:492] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:53.427316 17451 raft_consensus.cc:348] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.427426 17450 tablet_bootstrap.cc:654] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.427546 17451 raft_consensus.cc:374] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.427585 17406 raft_consensus.cc:482] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:53.427681 17406 raft_consensus.cc:504] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:53.427618 17451 raft_consensus.cc:729] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.427995 17451 consensus_queue.cc:260] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.428169 17406 leader_election.cc:288] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:53.428436 17451 ts_tablet_manager.cc:1398] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20240426 09:49:53.429772 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da96939413f24cebb13ff7744a6624f3" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:53.429921 9672 raft_consensus.cc:2454] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:53.430250 7208 leader_election.cc:302] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:53.430567 17406 raft_consensus.cc:2781] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:53.430630 17406 raft_consensus.cc:482] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:53.430670 17406 raft_consensus.cc:3037] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.430753 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da96939413f24cebb13ff7744a6624f3" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:53.430938 9417 raft_consensus.cc:2454] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:53.431164 17450 tablet_bootstrap.cc:492] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:53.431245 17450 ts_tablet_manager.cc:1367] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:53.431810 17406 raft_consensus.cc:504] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:53.432018 17451 tablet_bootstrap.cc:492] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:53.432166 17406 leader_election.cc:288] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:53.432788 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da96939413f24cebb13ff7744a6624f3" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:53.432904 9672 raft_consensus.cc:3037] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.433287 17451 tablet_bootstrap.cc:654] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.434021 9672 raft_consensus.cc:2454] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:53.434712 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da96939413f24cebb13ff7744a6624f3" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:53.434854 9416 raft_consensus.cc:3037] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.434823 17450 raft_consensus.cc:348] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:53.434929 17450 raft_consensus.cc:374] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.434963 17450 raft_consensus.cc:729] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.435089 17450 consensus_queue.cc:260] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:53.435369 17450 ts_tablet_manager.cc:1398] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:53.435876 9416 raft_consensus.cc:2454] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:53.436209 7208 leader_election.cc:302] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:53.436244 17451 tablet_bootstrap.cc:492] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:53.436357 17451 ts_tablet_manager.cc:1367] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:53.436901 17406 raft_consensus.cc:2781] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:53.436988 17406 raft_consensus.cc:686] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:53.437121 17406 consensus_queue.cc:237] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:53.437654 17451 raft_consensus.cc:348] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:53.437774 17451 raft_consensus.cc:374] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.437810 17451 raft_consensus.cc:729] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.437947 17451 consensus_queue.cc:260] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:53.438222 17451 ts_tablet_manager.cc:1398] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:53.439023 17451 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:53.439267 6735 catalog_manager.cc:5484] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:53.442062 17451 tablet_bootstrap.cc:654] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:53.444247 17451 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:53.444365 17451 ts_tablet_manager.cc:1367] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:49:53.444775 17451 raft_consensus.cc:348] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:53.444864 17451 raft_consensus.cc:374] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:53.444891 17451 raft_consensus.cc:729] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:53.444999 17451 consensus_queue.cc:260] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:53.445298 17451 ts_tablet_manager.cc:1398] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:53.470065 9265 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:53.470082 6991 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:53.481316 7255 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:53.483631 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:53.485570 9660 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:53.488016 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:53.491546 17461 raft_consensus.cc:482] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:53.491647 17461 raft_consensus.cc:504] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.492045 17461 leader_election.cc:288] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:53.492719 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd7063af4854e17bb2211e31033a89b" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:53.492743 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd7063af4854e17bb2211e31033a89b" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:53.492873 7013 raft_consensus.cc:2454] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:53.492872 9549 raft_consensus.cc:2454] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:53.493268 7077 leader_election.cc:302] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:53.493697 17461 raft_consensus.cc:2781] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:53.493770 17461 raft_consensus.cc:482] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:53.493830 17461 raft_consensus.cc:3037] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.494967 17461 raft_consensus.cc:504] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.495280 17461 leader_election.cc:288] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:53.495751 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd7063af4854e17bb2211e31033a89b" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:53.495858 9549 raft_consensus.cc:3037] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.495932 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecd7063af4854e17bb2211e31033a89b" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:53.496068 7013 raft_consensus.cc:3037] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.496778 9549 raft_consensus.cc:2454] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:53.497063 7013 raft_consensus.cc:2454] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:53.497143 7077 leader_election.cc:302] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:53.497342 17461 raft_consensus.cc:2781] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:53.497419 17461 raft_consensus.cc:686] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:53.497537 17461 consensus_queue.cc:237] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.503883 6735 catalog_manager.cc:5484] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:53.510368 9397 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:53.512970 9529 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:53.563689 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:53.568702 9672 raft_consensus.cc:1261] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:53.569016 9416 raft_consensus.cc:1261] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:53.572315 17490 consensus_queue.cc:1035] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:53.572957 17406 consensus_queue.cc:1035] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:53.576969 6735 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet da96939413f24cebb13ff7744a6624f3 (table rebalancer_dynamic_table [id=67cb01835dac48cabfac4e93e0b7f4a0]). Expected version 1 got 0
I20240426 09:49:53.577348 6732 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet da96939413f24cebb13ff7744a6624f3 (table rebalancer_dynamic_table [id=67cb01835dac48cabfac4e93e0b7f4a0]). Expected version 1 got 0
I20240426 09:49:53.582674 17497 tablet.cc:1722] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:53.583034 17499 tablet.cc:1722] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:53.583304 17497 tablet.cc:1620] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:53.583519 17499 tablet.cc:1620] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:53.586215 6732 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet da96939413f24cebb13ff7744a6624f3 (table rebalancer_dynamic_table [id=67cb01835dac48cabfac4e93e0b7f4a0]). Expected version 1 got 0
I20240426 09:49:53.587895 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=67cb01835dac48cabfac4e93e0b7f4a0] alter complete (version 1)
I20240426 09:49:53.590915 17498 tablet.cc:1722] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:53.591224 17498 tablet.cc:1620] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:53.601312 6880 consensus_queue.cc:237] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:53.602559 7142 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:49:53.602559 9549 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:49:53.602960 17500 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:53.603081 16574 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:53.604616 17507 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:53.605448 7142 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:53.608407 9549 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:53.608815 6732 catalog_manager.cc:5484] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 2 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:53.612761 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 53765d0ffcc24bb98aadc0cd0fe44af3 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:53.613440 17508 tablet_replica.cc:330] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:53.613605 17508 raft_consensus.cc:2229] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:53.613713 17508 raft_consensus.cc:2258] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:53.614234 17508 ts_tablet_manager.cc:1875] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:53.616210 17508 ts_tablet_manager.cc:1888] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240426 09:49:53.616297 17508 log.cc:1196] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/53765d0ffcc24bb98aadc0cd0fe44af3
I20240426 09:49:53.616775 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 53765d0ffcc24bb98aadc0cd0fe44af3 (table rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]) successfully deleted
I20240426 09:49:53.636081 17500 raft_consensus.cc:482] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:53.636211 17500 raft_consensus.cc:504] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:53.636582 17500 leader_election.cc:288] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:53.636931 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cec7dfa2ba1466d9d5fc6e55baf280b" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:53.636931 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cec7dfa2ba1466d9d5fc6e55baf280b" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:53.637090 7013 raft_consensus.cc:2454] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:53.637090 7142 raft_consensus.cc:2454] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:53.637355 6815 leader_election.cc:302] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:53.637512 17500 raft_consensus.cc:2781] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:53.637584 17500 raft_consensus.cc:482] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:53.637625 17500 raft_consensus.cc:3037] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.638638 17500 raft_consensus.cc:504] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:53.638945 17500 leader_election.cc:288] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:53.639259 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cec7dfa2ba1466d9d5fc6e55baf280b" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:53.639374 7013 raft_consensus.cc:3037] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.639343 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cec7dfa2ba1466d9d5fc6e55baf280b" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:53.639453 7142 raft_consensus.cc:3037] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.640403 7013 raft_consensus.cc:2454] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:53.640727 6815 leader_election.cc:302] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:53.640863 7142 raft_consensus.cc:2454] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:53.640883 17500 raft_consensus.cc:2781] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:53.641433 17500 raft_consensus.cc:686] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:53.641595 17500 consensus_queue.cc:237] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:53.643924 6735 catalog_manager.cc:5484] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:53.689512 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:53.689749 15237 raft_consensus.cc:482] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:53.689851 15237 raft_consensus.cc:504] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:53.690274 15237 leader_election.cc:288] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:53.690577 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348e90f5c63e4eda89b87fc103896302" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:53.690729 7142 raft_consensus.cc:2454] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:53.691044 6946 leader_election.cc:302] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441; no voters:
I20240426 09:49:53.691469 15237 raft_consensus.cc:2781] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:53.691466 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348e90f5c63e4eda89b87fc103896302" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:53.691530 15237 raft_consensus.cc:482] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:53.691567 15237 raft_consensus.cc:3037] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.691588 9549 raft_consensus.cc:2454] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:53.692641 15237 raft_consensus.cc:504] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:53.693600 15237 leader_election.cc:288] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:53.694025 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348e90f5c63e4eda89b87fc103896302" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:53.694152 9549 raft_consensus.cc:3037] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.694491 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "348e90f5c63e4eda89b87fc103896302" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:53.694595 7142 raft_consensus.cc:3037] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:53.695179 9549 raft_consensus.cc:2454] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:53.695601 6946 leader_election.cc:302] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:53.695811 15237 raft_consensus.cc:2781] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:53.696292 15237 raft_consensus.cc:686] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:53.696453 17497 tablet.cc:1722] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:53.696491 15237 consensus_queue.cc:237] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:53.696746 7142 raft_consensus.cc:2454] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:53.696784 17497 tablet.cc:1620] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: MemRowSet was empty: no flush needed.
I20240426 09:49:53.696979 17498 tablet.cc:1722] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:53.697288 17498 tablet.cc:1620] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:53.700382 17499 tablet.cc:1722] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:53.700726 17499 tablet.cc:1620] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:53.701689 6735 catalog_manager.cc:5484] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:53.702975 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=67cb01835dac48cabfac4e93e0b7f4a0] alter complete (version 2)
I20240426 09:49:53.805928 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:53.812176 17497 tablet.cc:1722] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:53.812584 17499 tablet.cc:1722] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:53.812882 17498 tablet.cc:1722] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:53.818640 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=67cb01835dac48cabfac4e93e0b7f4a0] alter complete (version 3)
I20240426 09:49:53.920619 6740 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:53.922395 6740 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:53.925554 6740 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet da96939413f24cebb13ff7744a6624f3
I20240426 09:49:53.926182 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet da96939413f24cebb13ff7744a6624f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:53 UTC) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:53.926878 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet da96939413f24cebb13ff7744a6624f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:53 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:53.927130 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet da96939413f24cebb13ff7744a6624f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:53 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:53.927187 17555 tablet_replica.cc:330] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:53.927328 17555 raft_consensus.cc:2229] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:53.927474 17555 raft_consensus.cc:2258] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:53.928068 17555 ts_tablet_manager.cc:1875] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:53.928915 17508 tablet_replica.cc:330] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:53.929061 17508 raft_consensus.cc:2229] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:53.929179 17508 raft_consensus.cc:2258] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:53.930974 17508 ts_tablet_manager.cc:1875] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:53.934607 17508 ts_tablet_manager.cc:1888] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:53.934891 17508 log.cc:1196] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/da96939413f24cebb13ff7744a6624f3
I20240426 09:49:53.935191 17508 ts_tablet_manager.cc:1909] T da96939413f24cebb13ff7744a6624f3 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:53.935181 17555 ts_tablet_manager.cc:1888] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:53.935240 17555 log.cc:1196] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/da96939413f24cebb13ff7744a6624f3
I20240426 09:49:53.935554 17555 ts_tablet_manager.cc:1909] T da96939413f24cebb13ff7744a6624f3 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:53.936857 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet da96939413f24cebb13ff7744a6624f3 (table rebalancer_dynamic_table_new_name [id=67cb01835dac48cabfac4e93e0b7f4a0]) successfully deleted
I20240426 09:49:53.937296 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet da96939413f24cebb13ff7744a6624f3 (table rebalancer_dynamic_table_new_name [id=67cb01835dac48cabfac4e93e0b7f4a0]) successfully deleted
I20240426 09:49:53.944644 17554 tablet_replica.cc:330] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:53.944839 17554 raft_consensus.cc:2229] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:53.945088 17554 raft_consensus.cc:2258] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:53.950119 17554 ts_tablet_manager.cc:1875] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:53.954070 17554 ts_tablet_manager.cc:1888] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:53.954859 17554 log.cc:1196] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/da96939413f24cebb13ff7744a6624f3
I20240426 09:49:53.955302 17554 ts_tablet_manager.cc:1909] T da96939413f24cebb13ff7744a6624f3 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting consensus metadata
I20240426 09:49:53.956158 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet da96939413f24cebb13ff7744a6624f3 (table rebalancer_dynamic_table_new_name [id=67cb01835dac48cabfac4e93e0b7f4a0]) successfully deleted
I20240426 09:49:53.978377 6991 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:53.981467 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:53.985662 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:53.989071 9660 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:53.994732 7255 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:53.997257 9397 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240426 09:49:54.006920 9265 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:54.016340 17461 consensus_queue.cc:1035] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.018793 17461 consensus_queue.cc:1035] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.029615 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:54.031322 9529 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:54.036051 6991 tablet_service.cc:1467] Processing CreateTablet for tablet b5bd4f060a1748b1a98880e3a824f8ba (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6dc5d8f6318440598913f2de7693b0db]), partition=
I20240426 09:49:54.036298 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5bd4f060a1748b1a98880e3a824f8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.038626 17572 tablet_bootstrap.cc:492] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:54.039059 9660 tablet_service.cc:1467] Processing CreateTablet for tablet b5bd4f060a1748b1a98880e3a824f8ba (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6dc5d8f6318440598913f2de7693b0db]), partition=
I20240426 09:49:54.039316 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5bd4f060a1748b1a98880e3a824f8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.041347 17573 tablet_bootstrap.cc:492] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:54.041486 9265 tablet_service.cc:1467] Processing CreateTablet for tablet b5bd4f060a1748b1a98880e3a824f8ba (DEFAULT_TABLE table=rebalancer_dynamic_table [id=6dc5d8f6318440598913f2de7693b0db]), partition=
I20240426 09:49:54.041718 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5bd4f060a1748b1a98880e3a824f8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.042263 17573 tablet_bootstrap.cc:654] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.044181 17575 tablet_bootstrap.cc:492] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:54.044179 17573 tablet_bootstrap.cc:492] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:54.044271 17573 ts_tablet_manager.cc:1367] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:54.044756 17573 raft_consensus.cc:348] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.044858 17573 raft_consensus.cc:374] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.044891 17573 raft_consensus.cc:729] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.045063 17573 consensus_queue.cc:260] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.047271 17573 ts_tablet_manager.cc:1398] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:49:54.048044 17575 tablet_bootstrap.cc:654] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.048568 17572 tablet_bootstrap.cc:654] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.049880 17572 tablet_bootstrap.cc:492] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:54.050000 17572 ts_tablet_manager.cc:1367] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20240426 09:49:54.050510 17572 raft_consensus.cc:348] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.050608 17572 raft_consensus.cc:374] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.050642 17572 raft_consensus.cc:729] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.050670 17575 tablet_bootstrap.cc:492] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:54.050773 17575 ts_tablet_manager.cc:1367] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:54.050819 17572 consensus_queue.cc:260] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.051146 17572 ts_tablet_manager.cc:1398] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.051204 17575 raft_consensus.cc:348] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.051329 17575 raft_consensus.cc:374] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.051359 17575 raft_consensus.cc:729] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.051470 17575 consensus_queue.cc:260] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.051724 17575 ts_tablet_manager.cc:1398] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.076967 17500 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.081444 17507 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.131982 9285 consensus_queue.cc:237] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.133641 9417 raft_consensus.cc:1261] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.133824 9549 raft_consensus.cc:1261] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.134097 9221 consensus_peers.cc:485] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab6450f48a36483883f8914075c224a8. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.134162 16894 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.134318 17072 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.135995 16894 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.141798 9417 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.144418 6733 catalog_manager.cc:5484] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.141847 9549 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.178439 7274 consensus_queue.cc:237] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.180029 9669 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:49:54.180045 6882 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240426 09:49:54.180135 7209 consensus_peers.cc:485] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4189112964fb425aac074352dd873415. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.180430 17517 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:49:54.180622 17406 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:49:54.183058 17517 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.183620 16849 raft_consensus.cc:482] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:54.183566 9669 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.183724 16849 raft_consensus.cc:504] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.184123 16849 leader_election.cc:288] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:54.184513 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5bd4f060a1748b1a98880e3a824f8ba" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:54.184661 9672 raft_consensus.cc:2454] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:54.184971 6946 leader_election.cc:302] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:54.185137 16849 raft_consensus.cc:2781] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:54.185184 16849 raft_consensus.cc:482] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:54.185220 16849 raft_consensus.cc:3037] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.185901 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5bd4f060a1748b1a98880e3a824f8ba" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:54.186031 9286 raft_consensus.cc:2454] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:54.186218 16849 raft_consensus.cc:504] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.186597 16849 leader_election.cc:288] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:54.186939 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5bd4f060a1748b1a98880e3a824f8ba" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:54.187044 9672 raft_consensus.cc:3037] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.186965 6882 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.187999 9672 raft_consensus.cc:2454] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:54.188277 6946 leader_election.cc:302] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:54.188727 15237 raft_consensus.cc:2781] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:54.188799 15237 raft_consensus.cc:686] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:54.188935 15237 consensus_queue.cc:237] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.189293 6733 catalog_manager.cc:5484] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.192999 6737 catalog_manager.cc:5484] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:54.193596 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5bd4f060a1748b1a98880e3a824f8ba" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:54.193717 9286 raft_consensus.cc:3037] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.194742 9286 raft_consensus.cc:2454] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:54.212795 6882 consensus_queue.cc:237] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
W20240426 09:49:54.214216 6815 consensus_peers.cc:485] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f8803aa38a646ebbdf5018ab7f65845. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.214284 9669 raft_consensus.cc:1261] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.214593 9549 raft_consensus.cc:1261] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.214668 17500 consensus_queue.cc:1035] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.215021 17500 consensus_queue.cc:1035] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.216746 16574 raft_consensus.cc:2932] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.217232 9547 raft_consensus.cc:2932] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.220183 6738 catalog_manager.cc:5484] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.220316 9672 raft_consensus.cc:2932] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.222900 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:54.232045 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.232045 7253 tablet_service.cc:1467] Processing CreateTablet for tablet b08039d1ab5e4b108ae21a24c507c876 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.232369 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.232331 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.232540 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b08039d1ab5e4b108ae21a24c507c876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.232571 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.233326 9396 tablet_service.cc:1467] Processing CreateTablet for tablet b08039d1ab5e4b108ae21a24c507c876 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.233615 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b08039d1ab5e4b108ae21a24c507c876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.234838 17649 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:54.235006 17648 tablet_bootstrap.cc:492] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:54.235030 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.235258 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.235900 17649 tablet_bootstrap.cc:654] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.236116 17648 tablet_bootstrap.cc:654] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.237000 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 937837e55579473087c0d0fcd8bcb001 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.237210 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937837e55579473087c0d0fcd8bcb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.237226 6992 tablet_service.cc:1467] Processing CreateTablet for tablet 937837e55579473087c0d0fcd8bcb001 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.237480 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937837e55579473087c0d0fcd8bcb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.237484 6991 tablet_service.cc:1467] Processing CreateTablet for tablet b08039d1ab5e4b108ae21a24c507c876 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.237758 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b08039d1ab5e4b108ae21a24c507c876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.237898 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 937837e55579473087c0d0fcd8bcb001 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.238094 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937837e55579473087c0d0fcd8bcb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.239624 17572 tablet_bootstrap.cc:492] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:54.240386 17648 tablet_bootstrap.cc:492] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:54.240458 17648 ts_tablet_manager.cc:1367] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:54.240759 17572 tablet_bootstrap.cc:654] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.240937 17648 raft_consensus.cc:348] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.241039 17648 raft_consensus.cc:374] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.241076 17648 raft_consensus.cc:729] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.241196 17648 consensus_queue.cc:260] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.241524 17648 ts_tablet_manager.cc:1398] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:54.241720 17648 tablet_bootstrap.cc:492] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:54.242187 17572 tablet_bootstrap.cc:492] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:54.242278 17572 ts_tablet_manager.cc:1367] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240426 09:49:54.242750 17572 raft_consensus.cc:348] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.242900 17572 raft_consensus.cc:374] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.242944 17572 raft_consensus.cc:729] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.242992 17648 tablet_bootstrap.cc:654] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.243081 17572 consensus_queue.cc:260] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.243389 17572 ts_tablet_manager.cc:1398] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:54.243600 17572 tablet_bootstrap.cc:492] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:54.244503 17572 tablet_bootstrap.cc:654] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.244870 17648 tablet_bootstrap.cc:492] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:54.244967 17648 ts_tablet_manager.cc:1367] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:54.245425 17648 raft_consensus.cc:348] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.245524 17648 raft_consensus.cc:374] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.245553 17648 raft_consensus.cc:729] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.245671 17648 consensus_queue.cc:260] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.245927 17648 ts_tablet_manager.cc:1398] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:54.246699 17572 tablet_bootstrap.cc:492] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:54.246815 17572 ts_tablet_manager.cc:1367] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240426 09:49:54.247260 17572 raft_consensus.cc:348] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.247354 17572 raft_consensus.cc:374] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.247390 17572 raft_consensus.cc:729] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.247515 17572 consensus_queue.cc:260] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.247691 17648 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:54.247785 17572 ts_tablet_manager.cc:1398] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.248601 17648 tablet_bootstrap.cc:654] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.249320 16849 raft_consensus.cc:482] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:54.249428 16849 raft_consensus.cc:504] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.249898 16849 leader_election.cc:288] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:54.251196 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937837e55579473087c0d0fcd8bcb001" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:54.251338 7275 raft_consensus.cc:2454] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:54.251341 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937837e55579473087c0d0fcd8bcb001" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
W20240426 09:49:54.251711 6948 leader_election.cc:341] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:54.251858 6948 leader_election.cc:302] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters: c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:49:54.252063 16849 raft_consensus.cc:2781] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:54.252127 16849 raft_consensus.cc:482] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:54.252187 16849 raft_consensus.cc:3037] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.252477 17648 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:54.252547 17648 ts_tablet_manager.cc:1367] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240426 09:49:54.252892 17650 tablet_bootstrap.cc:492] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:54.253245 16849 raft_consensus.cc:504] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.253594 16849 leader_election.cc:288] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:54.253536 17648 raft_consensus.cc:348] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.254129 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937837e55579473087c0d0fcd8bcb001" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
W20240426 09:49:54.254632 6948 leader_election.cc:341] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:54.254149 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937837e55579473087c0d0fcd8bcb001" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:54.254937 7275 raft_consensus.cc:3037] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.255204 17650 tablet_bootstrap.cc:654] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.255457 17649 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:54.255560 17649 ts_tablet_manager.cc:1367] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240426 09:49:54.255925 7275 raft_consensus.cc:2454] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:54.256053 17649 raft_consensus.cc:348] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.256157 17649 raft_consensus.cc:374] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.256187 17649 raft_consensus.cc:729] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.256284 6948 leader_election.cc:302] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters: c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:49:54.256323 17650 tablet_bootstrap.cc:492] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:54.256323 17649 consensus_queue.cc:260] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.256417 17650 ts_tablet_manager.cc:1367] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:54.256431 16849 raft_consensus.cc:2781] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:54.253655 17648 raft_consensus.cc:374] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.256484 17648 raft_consensus.cc:729] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.256506 16849 raft_consensus.cc:686] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:54.256609 17649 ts_tablet_manager.cc:1398] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.256613 17648 consensus_queue.cc:260] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.256636 16849 consensus_queue.cc:237] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.256794 17649 tablet_bootstrap.cc:492] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:54.256906 17648 ts_tablet_manager.cc:1398] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240426 09:49:54.256870 17650 raft_consensus.cc:348] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.256958 17650 raft_consensus.cc:374] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.256986 17650 raft_consensus.cc:729] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.257086 17650 consensus_queue.cc:260] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.257364 17650 ts_tablet_manager.cc:1398] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:54.257525 17650 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:54.257721 17649 tablet_bootstrap.cc:654] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.258436 17650 tablet_bootstrap.cc:654] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.260097 6737 catalog_manager.cc:5484] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.265863 17650 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:54.265969 17650 ts_tablet_manager.cc:1367] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240426 09:49:54.266309 17649 tablet_bootstrap.cc:492] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:54.266408 17649 ts_tablet_manager.cc:1367] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:54.266446 17650 raft_consensus.cc:348] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.266577 17650 raft_consensus.cc:374] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.266615 17650 raft_consensus.cc:729] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.266731 17650 consensus_queue.cc:260] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.266933 17649 raft_consensus.cc:348] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.267027 17649 raft_consensus.cc:374] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.267056 17649 raft_consensus.cc:729] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.267063 17650 ts_tablet_manager.cc:1398] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:54.267174 17649 consensus_queue.cc:260] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.268213 17649 ts_tablet_manager.cc:1398] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:54.270840 9669 consensus_queue.cc:237] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:54.272266 9547 raft_consensus.cc:1261] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:54.272545 9614 consensus_peers.cc:485] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdcd6105a2647738257638e1e1096c3. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.272763 17668 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:54.273347 9417 raft_consensus.cc:1261] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:54.273763 17668 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:54.275753 16791 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.276326 9417 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.278877 9547 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.279635 6737 catalog_manager.cc:5484] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.285081 15237 consensus_queue.cc:1035] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.287889 16849 consensus_queue.cc:1035] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.295826 17500 raft_consensus.cc:482] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:54.295934 17500 raft_consensus.cc:504] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.296399 17500 leader_election.cc:288] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:54.296742 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e78db4ae5cd4c40a3ed0478f8e9dee6" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:54.296886 9417 raft_consensus.cc:2454] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:54.297012 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e78db4ae5cd4c40a3ed0478f8e9dee6" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:54.297145 7275 raft_consensus.cc:2454] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:54.297190 6815 leader_election.cc:302] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:54.297483 17500 raft_consensus.cc:2781] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:54.297536 17500 raft_consensus.cc:482] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:54.297564 17500 raft_consensus.cc:3037] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.299011 17500 raft_consensus.cc:504] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.299350 17500 leader_election.cc:288] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:54.299664 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e78db4ae5cd4c40a3ed0478f8e9dee6" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:54.299778 9417 raft_consensus.cc:3037] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.299847 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e78db4ae5cd4c40a3ed0478f8e9dee6" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:54.299962 7275 raft_consensus.cc:3037] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.300786 9417 raft_consensus.cc:2454] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:54.300825 7275 raft_consensus.cc:2454] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:54.301263 6817 leader_election.cc:302] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:54.301720 17500 raft_consensus.cc:2781] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:54.301815 17500 raft_consensus.cc:686] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:54.301800 6734 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:54.301964 17500 consensus_queue.cc:237] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.307101 9669 raft_consensus.cc:1261] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.307193 9286 raft_consensus.cc:1261] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.307713 16849 consensus_queue.cc:1035] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.308249 7013 consensus_queue.cc:237] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.308641 6738 catalog_manager.cc:5484] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:54.309975 9285 raft_consensus.cc:1261] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.310025 15237 consensus_queue.cc:1035] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.310334 9547 raft_consensus.cc:1261] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.310421 6948 consensus_peers.cc:485] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e3624733f3444778b8656b3289d57dc. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.312166 15237 consensus_queue.cc:1035] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.312680 15237 consensus_queue.cc:1035] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.312935 6734 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet b5bd4f060a1748b1a98880e3a824f8ba (table rebalancer_dynamic_table [id=6dc5d8f6318440598913f2de7693b0db]). Expected version 1 got 0
I20240426 09:49:54.313347 17578 mvcc.cc:205] Tried to move back new op lower bound from 7021055976678293504 to 7021055976199262208. Current Snapshot: MvccSnapshot[applied={T|T < 7021055976678293504}]
I20240426 09:49:54.313772 17695 tablet.cc:1722] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:54.315625 17695 tablet.cc:1620] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:54.316836 6738 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet b5bd4f060a1748b1a98880e3a824f8ba (table rebalancer_dynamic_table [id=6dc5d8f6318440598913f2de7693b0db]). Expected version 1 got 0
I20240426 09:49:54.318306 17390 mvcc.cc:205] Tried to move back new op lower bound from 7021055976678293504 to 7021055976199262208. Current Snapshot: MvccSnapshot[applied={T|T < 7021055976678293504}]
I20240426 09:49:54.320111 17699 tablet.cc:1722] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:54.320458 17699 tablet.cc:1620] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:54.321205 6737 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet b5bd4f060a1748b1a98880e3a824f8ba (table rebalancer_dynamic_table [id=6dc5d8f6318440598913f2de7693b0db]). Expected version 1 got 0
I20240426 09:49:54.322175 6734 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet b5bd4f060a1748b1a98880e3a824f8ba (table rebalancer_dynamic_table [id=6dc5d8f6318440598913f2de7693b0db]). Expected version 1 got 0
I20240426 09:49:54.322499 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=6dc5d8f6318440598913f2de7693b0db] alter complete (version 1)
I20240426 09:49:54.318647 17696 tablet.cc:1722] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:54.323688 17696 tablet.cc:1620] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:54.331296 16849 raft_consensus.cc:2932] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.331655 9286 raft_consensus.cc:2932] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.333501 9549 raft_consensus.cc:2932] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.337280 6737 catalog_manager.cc:5484] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.358863 6882 consensus_queue.cc:237] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
W20240426 09:49:54.360471 6816 consensus_peers.cc:485] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e110e59d534c4bbd9fbceae078bb8286. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.360677 9417 raft_consensus.cc:1261] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.361079 17500 consensus_queue.cc:1035] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.361191 7275 raft_consensus.cc:1261] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.361723 17500 consensus_queue.cc:1035] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.363526 17507 raft_consensus.cc:2932] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.363941 9417 raft_consensus.cc:2932] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.367296 6738 catalog_manager.cc:5484] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.369033 7274 raft_consensus.cc:2932] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.381086 16895 raft_consensus.cc:482] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:54.381193 16895 raft_consensus.cc:504] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.381623 16895 leader_election.cc:288] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:54.381964 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08039d1ab5e4b108ae21a24c507c876" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:54.382007 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08039d1ab5e4b108ae21a24c507c876" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:54.382090 7274 raft_consensus.cc:2454] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:54.382145 7013 raft_consensus.cc:2454] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:54.382469 9350 leader_election.cc:302] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:54.382675 16895 raft_consensus.cc:2781] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:54.382727 16895 raft_consensus.cc:482] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:54.382802 16895 raft_consensus.cc:3037] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.383817 16895 raft_consensus.cc:504] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.384169 16895 leader_election.cc:288] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:54.384502 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08039d1ab5e4b108ae21a24c507c876" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:54.384617 7013 raft_consensus.cc:3037] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.384589 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08039d1ab5e4b108ae21a24c507c876" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:54.384707 7274 raft_consensus.cc:3037] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.385563 7013 raft_consensus.cc:2454] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:54.385568 7274 raft_consensus.cc:2454] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:54.385844 9350 leader_election.cc:302] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:54.385993 16895 raft_consensus.cc:2781] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:54.386081 16895 raft_consensus.cc:686] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:54.386205 16895 consensus_queue.cc:237] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.388772 6738 catalog_manager.cc:5484] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:54.407653 7274 consensus_queue.cc:237] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:54.409051 9669 raft_consensus.cc:1261] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.409142 6882 raft_consensus.cc:1261] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.409194 7208 consensus_peers.cc:485] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1a99c72632ab49a5b7b2c4a5b305b119. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.409489 17406 consensus_queue.cc:1035] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.409690 16735 consensus_queue.cc:1035] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.412375 17406 raft_consensus.cc:2932] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.412819 6882 raft_consensus.cc:2932] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.414952 9669 raft_consensus.cc:2932] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.415591 6738 catalog_manager.cc:5484] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.426896 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:54.433635 17695 tablet.cc:1722] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:54.433969 17695 tablet.cc:1620] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: MemRowSet was empty: no flush needed.
I20240426 09:49:54.435513 17696 tablet.cc:1722] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:54.435798 17696 tablet.cc:1620] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:54.437625 17699 tablet.cc:1722] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:54.437925 17699 tablet.cc:1620] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:54.441864 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=6dc5d8f6318440598913f2de7693b0db] alter complete (version 2)
I20240426 09:49:54.446446 9549 consensus_queue.cc:237] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:54.447888 9669 raft_consensus.cc:1261] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.447994 9482 consensus_peers.cc:485] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b32cacd1e7354933928c427f9646cafb. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.447983 9417 raft_consensus.cc:1261] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.448422 17387 consensus_queue.cc:1035] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.448642 16548 consensus_queue.cc:1035] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.450482 16548 raft_consensus.cc:2932] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.450918 9669 raft_consensus.cc:2932] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.453558 9417 raft_consensus.cc:2932] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.454108 6738 catalog_manager.cc:5484] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.473829 9672 consensus_queue.cc:237] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.475273 9549 raft_consensus.cc:1261] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:49:54.475343 7275 raft_consensus.cc:1261] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:49:54.475459 9616 consensus_peers.cc:485] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e306113f9874277a60468d9248d45ce. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.475672 16791 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:54.475842 17671 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:49:54.477659 16791 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.478224 7275 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.481491 6734 catalog_manager.cc:5484] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 3 to 4, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.482496 9549 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.500154 6882 consensus_queue.cc:237] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.501627 7013 raft_consensus.cc:1261] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.502000 9672 raft_consensus.cc:1261] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.502074 6817 consensus_peers.cc:485] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 22930f3d5b514c9884359f6c939c53e3. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.502362 17717 consensus_queue.cc:1035] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.502570 16574 consensus_queue.cc:1035] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.504276 17717 raft_consensus.cc:2932] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:54.504662 9672 raft_consensus.cc:2932] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:54.506497 17507 raft_consensus.cc:482] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:54.506608 17507 raft_consensus.cc:504] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.506978 17507 leader_election.cc:288] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:54.507687 6738 catalog_manager.cc:5484] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:54.508848 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937837e55579473087c0d0fcd8bcb001" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:54.508998 7275 raft_consensus.cc:2379] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c244e02f3cf847c0a5b69e1059f1071c in current term 1: Already voted for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in this term.
I20240426 09:49:54.508997 7013 raft_consensus.cc:2932] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:54.517102 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937837e55579473087c0d0fcd8bcb001" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:54.517490 6815 leader_election.cc:302] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c; no voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:54.518419 17507 raft_consensus.cc:3037] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.519559 17507 raft_consensus.cc:2726] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240426 09:49:54.534444 9672 consensus_queue.cc:237] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
W20240426 09:49:54.536063 9614 consensus_peers.cc:485] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 67e3f93c0a2944e38b7542d1439c907e. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.536135 6882 raft_consensus.cc:1261] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.536620 17668 consensus_queue.cc:1035] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.536746 9286 raft_consensus.cc:1261] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.537093 17668 consensus_queue.cc:1035] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.539367 16794 raft_consensus.cc:2932] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.539789 9286 raft_consensus.cc:2932] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.543092 6738 catalog_manager.cc:5484] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.545661 6734 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:54.548302 6882 raft_consensus.cc:2932] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:54.552248 17695 tablet.cc:1722] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:54.554921 17696 tablet.cc:1722] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:54.556115 17699 tablet.cc:1722] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:54.559053 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=6dc5d8f6318440598913f2de7693b0db] alter complete (version 3)
I20240426 09:49:54.567239 17824 ts_tablet_manager.cc:902] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:54.567816 17824 tablet_copy_client.cc:308] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:54.568245 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 4189112964fb425aac074352dd873415 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:40185)
I20240426 09:49:54.568346 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 4189112964fb425aac074352dd873415 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:40185: session id = a692602e1a2e4794877e6fb92a7b9962-4189112964fb425aac074352dd873415
I20240426 09:49:54.569240 7295 tablet_copy_source_session.cc:205] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:54.570449 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4189112964fb425aac074352dd873415. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.573661 17824 tablet_copy_client.cc:783] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:54.573930 17824 tablet_copy_client.cc:647] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:54.575156 17824 tablet_copy_client.cc:523] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:54.576267 17824 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:54.589330 9286 consensus_queue.cc:237] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:54.591200 9416 raft_consensus.cc:1261] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.592012 17072 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:49:54.592411 9219 consensus_peers.cc:485] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 279519370ee640fd9bbd3028008f1d5d. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.592458 7275 raft_consensus.cc:1261] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.593318 17072 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.595892 17601 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.598508 9416 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.599313 6737 catalog_manager.cc:5484] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.602259 7275 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:54.604233 17824 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:54.604668 17824 tablet_bootstrap.cc:492] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:54.604854 17824 ts_tablet_manager.cc:1367] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.029s user 0.027s sys 0.000s
I20240426 09:49:54.605350 17824 raft_consensus.cc:348] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.605453 17824 raft_consensus.cc:729] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.605583 17824 consensus_queue.cc:260] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.605859 17824 ts_tablet_manager.cc:1398] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.606709 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-4189112964fb425aac074352dd873415 received from {username='slave'} at 127.0.99.133:40185
I20240426 09:49:54.606909 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-4189112964fb425aac074352dd873415 on tablet 4189112964fb425aac074352dd873415 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:54.623065 6882 consensus_queue.cc:237] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.624585 7275 raft_consensus.cc:1261] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.624704 6815 consensus_peers.cc:485] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d04a7ec2c62437b879de204b5c885e9. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.624971 7013 raft_consensus.cc:1261] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.625169 16574 consensus_queue.cc:1035] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.625329 17507 consensus_queue.cc:1035] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.627027 16574 raft_consensus.cc:2932] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:54.627454 7275 raft_consensus.cc:2932] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:54.629472 7013 raft_consensus.cc:2932] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:54.630735 6738 catalog_manager.cc:5484] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:54.639075 17690 consensus_queue.cc:1035] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.643000 16849 consensus_queue.cc:1035] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.655113 9672 consensus_queue.cc:237] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.656380 6882 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.656384 9416 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.656569 9615 consensus_peers.cc:485] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2770cd77ebb642e3a52183a3871e33fa. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.656781 17668 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.656958 17777 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.659227 17668 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.659624 6882 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.661245 9416 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.661420 6734 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:54.661557 6734 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:54.662717 17878 ts_tablet_manager.cc:902] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:54.663178 6737 catalog_manager.cc:5484] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:54.663792 17878 tablet_copy_client.cc:308] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:54.664321 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 3f8803aa38a646ebbdf5018ab7f65845 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:34321)
I20240426 09:49:54.664425 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 3f8803aa38a646ebbdf5018ab7f65845 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:34321: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-3f8803aa38a646ebbdf5018ab7f65845
I20240426 09:49:54.665266 6902 tablet_copy_source_session.cc:205] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:54.666221 17878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8803aa38a646ebbdf5018ab7f65845. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.667016 6734 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet b5bd4f060a1748b1a98880e3a824f8ba
I20240426 09:49:54.667762 6991 tablet_service.cc:1514] Processing DeleteTablet for tablet b5bd4f060a1748b1a98880e3a824f8ba with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:54 UTC) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:54.668071 9265 tablet_service.cc:1514] Processing DeleteTablet for tablet b5bd4f060a1748b1a98880e3a824f8ba with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:54 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:54.668535 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet b5bd4f060a1748b1a98880e3a824f8ba with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:54 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:54.668576 17884 tablet_replica.cc:330] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:54.668716 17884 raft_consensus.cc:2229] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:54.668989 17884 raft_consensus.cc:2258] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:54.670053 17884 ts_tablet_manager.cc:1875] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:54.670240 17878 tablet_copy_client.cc:783] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:54.670545 17885 tablet_replica.cc:330] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:54.670730 17887 tablet_replica.cc:330] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:54.670830 17878 tablet_copy_client.cc:647] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:54.671311 17887 raft_consensus.cc:2229] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:54.671451 17887 raft_consensus.cc:2258] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:54.671656 17885 raft_consensus.cc:2229] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:54.671818 17885 raft_consensus.cc:2258] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:54.672479 17885 ts_tablet_manager.cc:1875] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:54.672935 17878 tablet_copy_client.cc:523] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:54.672935 17887 ts_tablet_manager.cc:1875] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:54.673349 17884 ts_tablet_manager.cc:1888] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:54.673415 17884 log.cc:1196] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/b5bd4f060a1748b1a98880e3a824f8ba
I20240426 09:49:54.673771 17884 ts_tablet_manager.cc:1909] T b5bd4f060a1748b1a98880e3a824f8ba P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting consensus metadata
I20240426 09:49:54.674218 17878 tablet_bootstrap.cc:492] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:54.674397 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet b5bd4f060a1748b1a98880e3a824f8ba (table rebalancer_dynamic_table_new_name [id=6dc5d8f6318440598913f2de7693b0db]) successfully deleted
I20240426 09:49:54.675487 17885 ts_tablet_manager.cc:1888] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:54.677592 17887 ts_tablet_manager.cc:1888] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:54.677729 17885 log.cc:1196] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/b5bd4f060a1748b1a98880e3a824f8ba
I20240426 09:49:54.677832 17887 log.cc:1196] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/b5bd4f060a1748b1a98880e3a824f8ba
I20240426 09:49:54.678315 17878 tablet_bootstrap.cc:492] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:54.678544 17887 ts_tablet_manager.cc:1909] T b5bd4f060a1748b1a98880e3a824f8ba P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:54.678612 17878 tablet_bootstrap.cc:492] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:49:54.678715 17885 ts_tablet_manager.cc:1909] T b5bd4f060a1748b1a98880e3a824f8ba P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:54.678835 17878 ts_tablet_manager.cc:1367] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.003s
I20240426 09:49:54.679287 17878 raft_consensus.cc:348] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:54.679381 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet b5bd4f060a1748b1a98880e3a824f8ba (table rebalancer_dynamic_table_new_name [id=6dc5d8f6318440598913f2de7693b0db]) successfully deleted
I20240426 09:49:54.679409 17878 raft_consensus.cc:729] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.679533 17878 consensus_queue.cc:260] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:54.679855 17878 ts_tablet_manager.cc:1398] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:54.679893 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet b5bd4f060a1748b1a98880e3a824f8ba (table rebalancer_dynamic_table_new_name [id=6dc5d8f6318440598913f2de7693b0db]) successfully deleted
I20240426 09:49:54.680367 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-3f8803aa38a646ebbdf5018ab7f65845 received from {username='slave'} at 127.0.99.130:34321
I20240426 09:49:54.680449 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-3f8803aa38a646ebbdf5018ab7f65845 on tablet 3f8803aa38a646ebbdf5018ab7f65845 with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:49:54.699311 9416 consensus_queue.cc:237] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.700810 7013 raft_consensus.cc:1261] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.700976 9352 consensus_peers.cc:485] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 536858c15b894b2f82efef38b8f51594. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.701153 9286 raft_consensus.cc:1261] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.701534 17511 consensus_queue.cc:1035] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.701857 17511 consensus_queue.cc:1035] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.703632 17511 raft_consensus.cc:2932] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:54.704033 9286 raft_consensus.cc:2932] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:54.705893 7013 raft_consensus.cc:2932] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:54.708479 6733 catalog_manager.cc:5484] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:54.711828 17909 ts_tablet_manager.cc:902] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:54.712376 17909 tablet_copy_client.cc:308] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:54.713398 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet bbdcd6105a2647738257638e1e1096c3 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:48359)
I20240426 09:49:54.713513 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet bbdcd6105a2647738257638e1e1096c3 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:48359: session id = 1aaf8befba984e199e96f53f46b50441-bbdcd6105a2647738257638e1e1096c3
I20240426 09:49:54.714524 9685 tablet_copy_source_session.cc:205] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:54.715499 17909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdcd6105a2647738257638e1e1096c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.716588 17908 ts_tablet_manager.cc:902] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:54.717178 17908 tablet_copy_client.cc:308] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:49:54.717607 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 4e3624733f3444778b8656b3289d57dc from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:51417)
I20240426 09:49:54.717700 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 4e3624733f3444778b8656b3289d57dc from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:51417: session id = c244e02f3cf847c0a5b69e1059f1071c-4e3624733f3444778b8656b3289d57dc
I20240426 09:49:54.718618 7033 tablet_copy_source_session.cc:205] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:54.720057 17908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3624733f3444778b8656b3289d57dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.720353 17909 tablet_copy_client.cc:783] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:54.720501 17909 tablet_copy_client.cc:647] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:54.721827 17909 tablet_copy_client.cc:523] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:54.722543 17908 tablet_copy_client.cc:783] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:54.722746 17908 tablet_copy_client.cc:647] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:54.724017 17908 tablet_copy_client.cc:523] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:54.725349 17908 tablet_bootstrap.cc:492] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:54.728771 17909 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:54.729243 17908 tablet_bootstrap.cc:492] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:54.729528 17908 tablet_bootstrap.cc:492] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:49:54.729700 17908 ts_tablet_manager.cc:1367] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20240426 09:49:54.730134 17908 raft_consensus.cc:348] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.730284 17908 raft_consensus.cc:729] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.730434 17908 consensus_queue.cc:260] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.730734 17908 ts_tablet_manager.cc:1398] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:54.731240 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-4e3624733f3444778b8656b3289d57dc received from {username='slave'} at 127.0.99.129:51417
I20240426 09:49:54.731338 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-4e3624733f3444778b8656b3289d57dc on tablet 4e3624733f3444778b8656b3289d57dc with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:49:54.734859 17908 ts_tablet_manager.cc:902] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:54.735260 17908 tablet_copy_client.cc:308] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:49:54.735813 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet ab6450f48a36483883f8914075c224a8 from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:36267)
I20240426 09:49:54.735908 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet ab6450f48a36483883f8914075c224a8 from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:36267: session id = c244e02f3cf847c0a5b69e1059f1071c-ab6450f48a36483883f8914075c224a8
I20240426 09:49:54.736856 9306 tablet_copy_source_session.cc:205] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:54.737978 17908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6450f48a36483883f8914075c224a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.740015 17908 tablet_copy_client.cc:783] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:54.740157 17908 tablet_copy_client.cc:647] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:54.741019 17909 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:54.741283 17909 tablet_bootstrap.cc:492] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:49:54.741458 17909 ts_tablet_manager.cc:1367] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20240426 09:49:54.741853 17909 raft_consensus.cc:348] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:54.741953 17909 raft_consensus.cc:729] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.742060 17909 consensus_queue.cc:260] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:54.742790 17909 ts_tablet_manager.cc:1398] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.743964 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-bbdcd6105a2647738257638e1e1096c3 received from {username='slave'} at 127.0.99.131:48359
I20240426 09:49:54.744040 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-bbdcd6105a2647738257638e1e1096c3 on tablet bbdcd6105a2647738257638e1e1096c3 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:54.744052 17908 tablet_copy_client.cc:523] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:54.745378 17908 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:54.747968 9286 consensus_queue.cc:237] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
W20240426 09:49:54.749886 9221 consensus_peers.cc:485] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aebef9b26fec446b8e71d4046768aedc. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.751093 17908 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:54.751255 9672 raft_consensus.cc:1261] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.751350 17908 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:49:54.751508 17908 ts_tablet_manager.cc:1367] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20240426 09:49:54.751780 9416 raft_consensus.cc:1261] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.751981 17908 raft_consensus.cc:348] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.752084 17908 raft_consensus.cc:729] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.752171 16894 consensus_queue.cc:1035] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.752205 17908 consensus_queue.cc:260] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:54.752514 17908 ts_tablet_manager.cc:1398] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:54.752501 16894 consensus_queue.cc:1035] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.753065 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-ab6450f48a36483883f8914075c224a8 received from {username='slave'} at 127.0.99.129:36267
I20240426 09:49:54.753137 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-ab6450f48a36483883f8914075c224a8 on tablet ab6450f48a36483883f8914075c224a8 with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:49:54.762943 16894 raft_consensus.cc:2932] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.765081 9672 raft_consensus.cc:2932] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.767339 9416 raft_consensus.cc:2932] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.769285 6733 catalog_manager.cc:5484] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:54.770478 6737 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:54.777073 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 1e2cef7d173846cc895d78cc27f1103e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f604704ce5ab44338600852a208d21fa]), partition=
I20240426 09:49:54.777364 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e2cef7d173846cc895d78cc27f1103e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.777961 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 1e2cef7d173846cc895d78cc27f1103e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f604704ce5ab44338600852a208d21fa]), partition=
I20240426 09:49:54.778206 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e2cef7d173846cc895d78cc27f1103e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.780053 17948 tablet_bootstrap.cc:492] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:54.781800 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 1e2cef7d173846cc895d78cc27f1103e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=f604704ce5ab44338600852a208d21fa]), partition=
I20240426 09:49:54.782012 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e2cef7d173846cc895d78cc27f1103e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.782478 17949 tablet_bootstrap.cc:492] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:54.784341 17948 tablet_bootstrap.cc:654] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.784597 17950 tablet_bootstrap.cc:492] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:54.785567 17950 tablet_bootstrap.cc:654] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.785620 17949 tablet_bootstrap.cc:654] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.786707 17950 tablet_bootstrap.cc:492] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:54.786816 17950 ts_tablet_manager.cc:1367] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:54.787396 17950 raft_consensus.cc:348] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.787501 17950 raft_consensus.cc:374] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.787546 17950 raft_consensus.cc:729] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.787675 17950 consensus_queue.cc:260] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.787945 17950 ts_tablet_manager.cc:1398] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.788102 17948 tablet_bootstrap.cc:492] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:54.788185 17948 ts_tablet_manager.cc:1367] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:54.788589 17948 raft_consensus.cc:348] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.788703 17948 raft_consensus.cc:374] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.788733 17948 raft_consensus.cc:729] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.788856 17948 consensus_queue.cc:260] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.789186 17948 ts_tablet_manager.cc:1398] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.789415 17949 tablet_bootstrap.cc:492] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:54.789494 17949 ts_tablet_manager.cc:1367] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:54.789903 17949 raft_consensus.cc:348] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.789994 17949 raft_consensus.cc:374] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.790022 17949 raft_consensus.cc:729] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.790151 17949 consensus_queue.cc:260] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.790421 17949 ts_tablet_manager.cc:1398] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.805296 9286 consensus_queue.cc:237] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.806723 7143 raft_consensus.cc:1261] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.806717 9672 raft_consensus.cc:1261] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.806777 9219 consensus_peers.cc:485] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 638b15a2c9ea42c3b86a985ccae550a7. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.807147 17598 consensus_queue.cc:1035] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.807336 16894 consensus_queue.cc:1035] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.809221 17598 raft_consensus.cc:2932] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:54.809613 9672 raft_consensus.cc:2932] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:54.812523 7143 raft_consensus.cc:2932] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:54.812719 6738 catalog_manager.cc:5484] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:54.832684 7275 consensus_queue.cc:237] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.834225 9286 raft_consensus.cc:1261] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:54.834757 7143 raft_consensus.cc:1261] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:49:54.834879 7208 consensus_peers.cc:485] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d83c01e6dc6d4b5b9365243058f459a1. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.835108 17517 consensus_queue.cc:1035] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.835309 17490 consensus_queue.cc:1035] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:49:54.837309 17517 raft_consensus.cc:2932] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:54.837671 7142 raft_consensus.cc:2932] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:54.839497 9286 raft_consensus.cc:2932] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:54.841189 6738 catalog_manager.cc:5484] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:54.859491 17905 consensus_queue.cc:1035] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.859913 17717 raft_consensus.cc:482] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:54.860024 17717 raft_consensus.cc:504] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.860424 17717 leader_election.cc:288] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:54.861176 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2cef7d173846cc895d78cc27f1103e" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:54.861310 9416 raft_consensus.cc:2454] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:54.861287 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2cef7d173846cc895d78cc27f1103e" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:54.861414 9672 raft_consensus.cc:2454] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 0.
I20240426 09:49:54.862912 6815 leader_election.cc:302] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:54.862928 17905 consensus_queue.cc:1035] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.863207 17717 raft_consensus.cc:2781] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:54.863265 17717 raft_consensus.cc:482] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:54.863310 17717 raft_consensus.cc:3037] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.864554 17717 raft_consensus.cc:504] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:54.864925 17717 leader_election.cc:288] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:54.865229 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2cef7d173846cc895d78cc27f1103e" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:54.865335 9672 raft_consensus.cc:3037] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.865465 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2cef7d173846cc895d78cc27f1103e" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:54.865566 9416 raft_consensus.cc:3037] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.866317 9672 raft_consensus.cc:2454] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:54.866457 9416 raft_consensus.cc:2454] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 1.
I20240426 09:49:54.866917 6815 leader_election.cc:302] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:54.868330 7275 consensus_queue.cc:237] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:54.869716 7013 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:49:54.869742 9286 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:49:54.870169 16574 raft_consensus.cc:2781] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:54.870129 16735 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:49:54.870252 16574 raft_consensus.cc:686] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:54.870393 16574 consensus_queue.cc:237] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
W20240426 09:49:54.870474 7208 consensus_peers.cc:485] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b2d42c52b7a94e32ab4cdad78ce884ee. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:49:54.870898 16735 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:49:54.872109 17717 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.873174 17490 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:54.873888 7013 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:54.875124 9286 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:54.875701 6740 catalog_manager.cc:5484] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 0 to 1, leader changed from <none> to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:54.877190 6734 catalog_manager.cc:5484] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:54.883087 17507 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:54.907346 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:54.914352 9529 tablet_service.cc:1467] Processing CreateTablet for tablet d6981747bd8c44d2bb56f3d5bb56dbb1 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.914701 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6981747bd8c44d2bb56f3d5bb56dbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.914870 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 4613f57999fa4a6baf5ccfd102f5c73b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.915120 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4613f57999fa4a6baf5ccfd102f5c73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.917146 18025 tablet_bootstrap.cc:492] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:54.917232 17949 tablet_bootstrap.cc:492] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:54.917980 7255 tablet_service.cc:1467] Processing CreateTablet for tablet c0a2aef96c3845b7bb6357b9914fe7f8 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.918215 18025 tablet_bootstrap.cc:654] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.918289 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0a2aef96c3845b7bb6357b9914fe7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.918920 17949 tablet_bootstrap.cc:654] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.919260 18025 tablet_bootstrap.cc:492] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:54.919348 18025 ts_tablet_manager.cc:1367] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:54.919662 9529 tablet_service.cc:1467] Processing CreateTablet for tablet c0a2aef96c3845b7bb6357b9914fe7f8 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.919692 9265 tablet_service.cc:1467] Processing CreateTablet for tablet d6981747bd8c44d2bb56f3d5bb56dbb1 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.919773 18025 raft_consensus.cc:348] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.919878 18025 raft_consensus.cc:374] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.919943 18025 raft_consensus.cc:729] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.919955 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6981747bd8c44d2bb56f3d5bb56dbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.919867 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0a2aef96c3845b7bb6357b9914fe7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.920073 18025 consensus_queue.cc:260] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.920367 18025 ts_tablet_manager.cc:1398] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.920711 18028 tablet_bootstrap.cc:492] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:54.921186 7123 tablet_service.cc:1467] Processing CreateTablet for tablet c0a2aef96c3845b7bb6357b9914fe7f8 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.921458 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0a2aef96c3845b7bb6357b9914fe7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.921751 18028 tablet_bootstrap.cc:654] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.923242 18029 tablet_bootstrap.cc:492] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:54.923844 18031 tablet_bootstrap.cc:492] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:54.923977 9264 tablet_service.cc:1467] Processing CreateTablet for tablet 4613f57999fa4a6baf5ccfd102f5c73b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.924101 18025 tablet_bootstrap.cc:492] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:54.924207 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4613f57999fa4a6baf5ccfd102f5c73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.924496 17949 tablet_bootstrap.cc:492] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:54.924585 17949 ts_tablet_manager.cc:1367] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:54.925029 18025 tablet_bootstrap.cc:654] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.925086 17949 raft_consensus.cc:348] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.925179 17949 raft_consensus.cc:374] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.925216 17949 raft_consensus.cc:729] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.925356 17949 consensus_queue.cc:260] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.925683 17949 ts_tablet_manager.cc:1398] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.914352 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 4613f57999fa4a6baf5ccfd102f5c73b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.927863 18031 tablet_bootstrap.cc:654] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.927949 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4613f57999fa4a6baf5ccfd102f5c73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.928182 18029 tablet_bootstrap.cc:654] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.929248 18028 tablet_bootstrap.cc:492] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:54.929316 18028 ts_tablet_manager.cc:1367] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:54.929642 18028 raft_consensus.cc:348] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.930296 18028 raft_consensus.cc:374] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.930490 17878 ts_tablet_manager.cc:902] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:54.930958 17878 tablet_copy_client.cc:308] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:54.931555 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 1a99c72632ab49a5b7b2c4a5b305b119 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:36325)
I20240426 09:49:54.931636 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 1a99c72632ab49a5b7b2c4a5b305b119 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:36325: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-1a99c72632ab49a5b7b2c4a5b305b119
I20240426 09:49:54.930368 18028 raft_consensus.cc:729] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.931974 18039 ts_tablet_manager.cc:902] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:54.932003 18028 consensus_queue.cc:260] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.932375 18028 ts_tablet_manager.cc:1398] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:54.932587 7295 tablet_copy_source_session.cc:205] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:54.932608 18039 tablet_copy_client.cc:308] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:54.933105 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 22930f3d5b514c9884359f6c939c53e3 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:36623)
I20240426 09:49:54.933213 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 22930f3d5b514c9884359f6c939c53e3 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:36623: session id = e3394a4be9ea4def8aa39b333c82c5ff-22930f3d5b514c9884359f6c939c53e3
I20240426 09:49:54.914875 9660 tablet_service.cc:1467] Processing CreateTablet for tablet d6981747bd8c44d2bb56f3d5bb56dbb1 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:54.933811 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6981747bd8c44d2bb56f3d5bb56dbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.933996 18025 tablet_bootstrap.cc:492] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:54.934067 18025 ts_tablet_manager.cc:1367] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:54.934190 6902 tablet_copy_source_session.cc:205] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:54.934486 18025 raft_consensus.cc:348] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.934572 18025 raft_consensus.cc:374] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.934598 18025 raft_consensus.cc:729] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.934691 18025 consensus_queue.cc:260] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.934953 18025 ts_tablet_manager.cc:1398] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.935082 18025 tablet_bootstrap.cc:492] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:54.935868 18025 tablet_bootstrap.cc:654] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.935964 17949 tablet_bootstrap.cc:492] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:54.936862 17949 tablet_bootstrap.cc:654] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.936918 18025 tablet_bootstrap.cc:492] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:54.936980 18025 ts_tablet_manager.cc:1367] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:54.937323 18025 raft_consensus.cc:348] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.937402 18025 raft_consensus.cc:374] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.937428 18025 raft_consensus.cc:729] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.937538 18025 consensus_queue.cc:260] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.937757 18025 ts_tablet_manager.cc:1398] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.938329 17878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a99c72632ab49a5b7b2c4a5b305b119. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.941702 18029 tablet_bootstrap.cc:492] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:54.941804 18029 ts_tablet_manager.cc:1367] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240426 09:49:54.942302 18029 raft_consensus.cc:348] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.942384 18029 raft_consensus.cc:374] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.942404 18029 raft_consensus.cc:729] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.942564 18029 consensus_queue.cc:260] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.942868 18029 ts_tablet_manager.cc:1398] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.943203 17878 tablet_copy_client.cc:783] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:54.943361 17878 tablet_copy_client.cc:647] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:54.944726 17878 tablet_copy_client.cc:523] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:54.945976 17878 tablet_bootstrap.cc:492] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:54.946981 18029 tablet_bootstrap.cc:492] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:54.947968 18029 tablet_bootstrap.cc:654] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:54.953639 18029 tablet_bootstrap.cc:492] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:54.953815 18029 ts_tablet_manager.cc:1367] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:54.954471 18039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22930f3d5b514c9884359f6c939c53e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.955178 18029 raft_consensus.cc:348] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.955380 18029 raft_consensus.cc:374] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.955456 18029 raft_consensus.cc:729] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.955654 18029 consensus_queue.cc:260] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.956274 18031 tablet_bootstrap.cc:492] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:54.956370 18031 ts_tablet_manager.cc:1367] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20240426 09:49:54.956619 17878 tablet_bootstrap.cc:492] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:54.956852 17878 tablet_bootstrap.cc:492] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:49:54.956815 18031 raft_consensus.cc:348] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.956943 18031 raft_consensus.cc:374] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.956980 18031 raft_consensus.cc:729] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.957010 17878 ts_tablet_manager.cc:1367] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.003s
I20240426 09:49:54.956864 18029 ts_tablet_manager.cc:1398] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:54.957106 18031 consensus_queue.cc:260] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.957391 18031 ts_tablet_manager.cc:1398] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:54.957412 17878 raft_consensus.cc:348] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:54.957513 17878 raft_consensus.cc:729] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.957623 17878 consensus_queue.cc:260] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:54.961584 17824 ts_tablet_manager.cc:902] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:54.962127 17824 tablet_copy_client.cc:308] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:54.962180 18039 tablet_copy_client.cc:783] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:54.962589 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet e110e59d534c4bbd9fbceae078bb8286 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:54839)
I20240426 09:49:54.962692 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet e110e59d534c4bbd9fbceae078bb8286 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:54839: session id = a692602e1a2e4794877e6fb92a7b9962-e110e59d534c4bbd9fbceae078bb8286
I20240426 09:49:54.962924 17878 ts_tablet_manager.cc:1398] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20240426 09:49:54.963320 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-1a99c72632ab49a5b7b2c4a5b305b119 received from {username='slave'} at 127.0.99.130:36325
I20240426 09:49:54.963382 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-1a99c72632ab49a5b7b2c4a5b305b119 on tablet 1a99c72632ab49a5b7b2c4a5b305b119 with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:49:54.963793 6902 tablet_copy_source_session.cc:205] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:54.964556 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e110e59d534c4bbd9fbceae078bb8286. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:54.964763 18039 tablet_copy_client.cc:647] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:54.966698 17824 tablet_copy_client.cc:783] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:54.968691 17824 tablet_copy_client.cc:647] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:54.969085 18039 tablet_copy_client.cc:523] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:54.970517 18039 tablet_bootstrap.cc:492] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:54.970656 17824 tablet_copy_client.cc:523] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:54.971652 17668 raft_consensus.cc:482] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:54.971782 17668 raft_consensus.cc:504] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.972172 17824 tablet_bootstrap.cc:492] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:54.972270 17668 leader_election.cc:288] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:54.972476 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4613f57999fa4a6baf5ccfd102f5c73b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:54.972597 9286 raft_consensus.cc:2454] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:54.972679 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4613f57999fa4a6baf5ccfd102f5c73b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:54.972808 9549 raft_consensus.cc:2454] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:54.972851 9615 leader_election.cc:302] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:54.973006 17668 raft_consensus.cc:2781] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:54.973052 17668 raft_consensus.cc:482] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:54.973085 17668 raft_consensus.cc:3037] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.974104 17668 raft_consensus.cc:504] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.974458 17668 leader_election.cc:288] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:54.974558 17949 tablet_bootstrap.cc:492] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:54.974648 17949 ts_tablet_manager.cc:1367] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.000s
I20240426 09:49:54.974895 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4613f57999fa4a6baf5ccfd102f5c73b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:54.974989 9286 raft_consensus.cc:3037] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.975093 17949 raft_consensus.cc:348] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.975174 17949 raft_consensus.cc:374] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:54.975219 17949 raft_consensus.cc:729] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.975337 17949 consensus_queue.cc:260] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.975601 17949 ts_tablet_manager.cc:1398] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.975922 9286 raft_consensus.cc:2454] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:54.976806 9615 leader_election.cc:302] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:54.977123 17777 raft_consensus.cc:2781] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:54.977192 17777 raft_consensus.cc:686] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:54.977331 17777 consensus_queue.cc:237] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:54.979475 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4613f57999fa4a6baf5ccfd102f5c73b" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:54.979594 9549 raft_consensus.cc:3037] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.980618 9549 raft_consensus.cc:2454] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:54.984771 6740 catalog_manager.cc:5484] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:54.985685 18039 tablet_bootstrap.cc:492] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:54.985985 18039 tablet_bootstrap.cc:492] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:49:54.986166 18039 ts_tablet_manager.cc:1367] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20240426 09:49:54.986608 17490 raft_consensus.cc:482] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:54.986626 18039 raft_consensus.cc:348] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.986730 18039 raft_consensus.cc:729] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.986686 17490 raft_consensus.cc:504] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.986879 18039 consensus_queue.cc:260] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.987052 17490 leader_election.cc:288] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:54.987196 18039 ts_tablet_manager.cc:1398] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:54.987587 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0a2aef96c3845b7bb6357b9914fe7f8" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:54.987627 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0a2aef96c3845b7bb6357b9914fe7f8" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:54.987728 7142 raft_consensus.cc:2454] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:54.987751 9549 raft_consensus.cc:2454] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:54.988108 7208 leader_election.cc:302] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:54.988373 17490 raft_consensus.cc:2781] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:54.988426 17490 raft_consensus.cc:482] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:54.988458 17490 raft_consensus.cc:3037] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.989286 17824 tablet_bootstrap.cc:492] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:54.989529 17490 raft_consensus.cc:504] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.989593 17824 tablet_bootstrap.cc:492] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:54.989809 17824 ts_tablet_manager.cc:1367] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.004s
I20240426 09:49:54.989852 17490 leader_election.cc:288] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:54.990345 17824 raft_consensus.cc:348] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.990439 17824 raft_consensus.cc:729] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:54.990408 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-22930f3d5b514c9884359f6c939c53e3 received from {username='slave'} at 127.0.99.132:36623
I20240426 09:49:54.990504 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-22930f3d5b514c9884359f6c939c53e3 on tablet 22930f3d5b514c9884359f6c939c53e3 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:54.990599 17824 consensus_queue.cc:260] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:54.991421 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0a2aef96c3845b7bb6357b9914fe7f8" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:54.991439 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0a2aef96c3845b7bb6357b9914fe7f8" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:54.991559 9549 raft_consensus.cc:3037] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.991549 7142 raft_consensus.cc:3037] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:54.992072 17824 ts_tablet_manager.cc:1398] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:49:54.992514 9549 raft_consensus.cc:2454] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:54.992565 7142 raft_consensus.cc:2454] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:54.992879 7208 leader_election.cc:302] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:54.993057 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-e110e59d534c4bbd9fbceae078bb8286 received from {username='slave'} at 127.0.99.133:54839
I20240426 09:49:54.993134 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-e110e59d534c4bbd9fbceae078bb8286 on tablet e110e59d534c4bbd9fbceae078bb8286 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:54.993662 17490 raft_consensus.cc:2781] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:54.993739 17490 raft_consensus.cc:686] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:54.993849 17490 consensus_queue.cc:237] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:54.996829 6733 catalog_manager.cc:5484] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:55.004446 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:55.008870 9672 raft_consensus.cc:1261] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:55.008949 9416 raft_consensus.cc:1261] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:55.009279 17717 consensus_queue.cc:1035] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.009405 17507 consensus_queue.cc:1035] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.012459 6737 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 1e2cef7d173846cc895d78cc27f1103e (table rebalancer_dynamic_table [id=f604704ce5ab44338600852a208d21fa]). Expected version 1 got 0
I20240426 09:49:55.013350 17909 ts_tablet_manager.cc:902] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:55.013969 17909 tablet_copy_client.cc:308] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:49:55.014209 6737 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet 1e2cef7d173846cc895d78cc27f1103e (table rebalancer_dynamic_table [id=f604704ce5ab44338600852a208d21fa]). Expected version 1 got 0
I20240426 09:49:55.015640 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet 279519370ee640fd9bbd3028008f1d5d from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:36823)
I20240426 09:49:55.015735 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet 279519370ee640fd9bbd3028008f1d5d from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:36823: session id = 1aaf8befba984e199e96f53f46b50441-279519370ee640fd9bbd3028008f1d5d
I20240426 09:49:55.016618 9306 tablet_copy_source_session.cc:205] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.017710 17909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279519370ee640fd9bbd3028008f1d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.017992 17867 mvcc.cc:205] Tried to move back new op lower bound from 7021055979551363072 to 7021055978990526464. Current Snapshot: MvccSnapshot[applied={T|T < 7021055979551363072}]
I20240426 09:49:55.019899 17909 tablet_copy_client.cc:783] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.020052 17909 tablet_copy_client.cc:647] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.020303 17699 tablet.cc:1722] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:55.020586 17699 tablet.cc:1620] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:55.021600 6733 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet 1e2cef7d173846cc895d78cc27f1103e (table rebalancer_dynamic_table [id=f604704ce5ab44338600852a208d21fa]). Expected version 1 got 0
I20240426 09:49:55.022162 6737 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet 1e2cef7d173846cc895d78cc27f1103e (table rebalancer_dynamic_table [id=f604704ce5ab44338600852a208d21fa]). Expected version 1 got 0
I20240426 09:49:55.029284 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=f604704ce5ab44338600852a208d21fa] alter complete (version 1)
I20240426 09:49:55.042174 18088 tablet.cc:1722] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:55.042642 18088 tablet.cc:1620] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:55.042680 18098 ts_tablet_manager.cc:902] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:55.043942 18098 tablet_copy_client.cc:308] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:49:55.044422 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 0d04a7ec2c62437b879de204b5c885e9 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:55679)
I20240426 09:49:55.044541 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 0d04a7ec2c62437b879de204b5c885e9 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:55679: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-0d04a7ec2c62437b879de204b5c885e9
I20240426 09:49:55.045575 6902 tablet_copy_source_session.cc:205] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.045826 17909 tablet_copy_client.cc:523] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.047333 17909 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:55.047804 18098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d04a7ec2c62437b879de204b5c885e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.048265 18089 tablet.cc:1722] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:55.048671 18089 tablet.cc:1620] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:55.051095 18098 tablet_copy_client.cc:783] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.051295 18098 tablet_copy_client.cc:647] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.058190 17387 raft_consensus.cc:482] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:55.058321 17387 raft_consensus.cc:504] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.059798 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6981747bd8c44d2bb56f3d5bb56dbb1" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:55.059863 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6981747bd8c44d2bb56f3d5bb56dbb1" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:55.059948 9285 raft_consensus.cc:2454] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:55.060005 9672 raft_consensus.cc:2454] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:55.058751 17387 leader_election.cc:288] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:55.060587 9482 leader_election.cc:302] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:55.064234 17387 raft_consensus.cc:2781] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:55.064303 17387 raft_consensus.cc:482] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:55.064342 17387 raft_consensus.cc:3037] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.065464 17387 raft_consensus.cc:504] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.065851 17387 leader_election.cc:288] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:55.066421 18098 tablet_copy_client.cc:523] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.066697 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6981747bd8c44d2bb56f3d5bb56dbb1" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:55.066846 9672 raft_consensus.cc:3037] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.067131 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6981747bd8c44d2bb56f3d5bb56dbb1" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:55.067252 9285 raft_consensus.cc:3037] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.067899 9672 raft_consensus.cc:2454] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:55.068245 9285 raft_consensus.cc:2454] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:55.070012 9482 leader_election.cc:302] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:55.070168 17387 raft_consensus.cc:2781] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:55.070245 17387 raft_consensus.cc:686] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:55.070376 17387 consensus_queue.cc:237] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.074328 6734 catalog_manager.cc:5484] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:55.083451 18104 ts_tablet_manager.cc:902] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:55.083822 18098 tablet_bootstrap.cc:492] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:55.086428 18104 tablet_copy_client.cc:308] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:49:55.086891 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet b32cacd1e7354933928c427f9646cafb from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:45949)
I20240426 09:49:55.086993 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet b32cacd1e7354933928c427f9646cafb from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:45949: session id = 1aaf8befba984e199e96f53f46b50441-b32cacd1e7354933928c427f9646cafb
I20240426 09:49:55.087913 9569 tablet_copy_source_session.cc:205] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.089776 18104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b32cacd1e7354933928c427f9646cafb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.089910 18098 tablet_bootstrap.cc:492] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.090690 18098 tablet_bootstrap.cc:492] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:55.090884 9285 raft_consensus.cc:1203] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240426 09:49:55.091065 18098 ts_tablet_manager.cc:1367] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20240426 09:49:55.092573 18098 raft_consensus.cc:348] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:55.092701 18098 raft_consensus.cc:729] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.092871 18098 consensus_queue.cc:260] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:55.094349 18098 ts_tablet_manager.cc:1398] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:55.094831 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-0d04a7ec2c62437b879de204b5c885e9 received from {username='slave'} at 127.0.99.135:55679
I20240426 09:49:55.094908 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-0d04a7ec2c62437b879de204b5c885e9 on tablet 0d04a7ec2c62437b879de204b5c885e9 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:55.096700 17909 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.096963 17909 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:49:55.097126 17909 ts_tablet_manager.cc:1367] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.003s
I20240426 09:49:55.097242 17908 ts_tablet_manager.cc:902] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:55.097610 17909 raft_consensus.cc:348] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:55.097729 17909 raft_consensus.cc:729] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.097738 17908 tablet_copy_client.cc:308] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:55.097857 17909 consensus_queue.cc:260] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:55.098168 17909 ts_tablet_manager.cc:1398] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:55.098179 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 8e306113f9874277a60468d9248d45ce from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:42751)
I20240426 09:49:55.098291 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 8e306113f9874277a60468d9248d45ce from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:42751: session id = c244e02f3cf847c0a5b69e1059f1071c-8e306113f9874277a60468d9248d45ce
I20240426 09:49:55.099134 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-279519370ee640fd9bbd3028008f1d5d received from {username='slave'} at 127.0.99.131:36823
I20240426 09:49:55.099228 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-279519370ee640fd9bbd3028008f1d5d on tablet 279519370ee640fd9bbd3028008f1d5d with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:55.099277 9685 tablet_copy_source_session.cc:205] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.100955 18104 tablet_copy_client.cc:783] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.101143 18104 tablet_copy_client.cc:647] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.101409 17908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e306113f9874277a60468d9248d45ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.103477 18104 tablet_copy_client.cc:523] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.104475 17908 tablet_copy_client.cc:783] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.104638 17908 tablet_copy_client.cc:647] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.105361 18104 tablet_bootstrap.cc:492] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:55.106225 17908 tablet_copy_client.cc:523] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.108484 17908 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:55.113317 17908 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.113623 17908 tablet_bootstrap.cc:492] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:49:55.113777 17908 ts_tablet_manager.cc:1367] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:49:55.114130 18104 tablet_bootstrap.cc:492] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.114241 17908 raft_consensus.cc:348] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.114364 17908 raft_consensus.cc:729] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.114388 18104 tablet_bootstrap.cc:492] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:49:55.114533 18104 ts_tablet_manager.cc:1367] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20240426 09:49:55.114496 17908 consensus_queue.cc:260] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.115366 18104 raft_consensus.cc:348] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:55.115475 18104 raft_consensus.cc:729] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.116446 18104 consensus_queue.cc:260] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:55.116878 18104 ts_tablet_manager.cc:1398] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:55.117403 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-b32cacd1e7354933928c427f9646cafb received from {username='slave'} at 127.0.99.131:45949
I20240426 09:49:55.117482 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-b32cacd1e7354933928c427f9646cafb on tablet b32cacd1e7354933928c427f9646cafb with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:49:55.122146 6882 raft_consensus.cc:1203] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.127197 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:55.127278 17908 ts_tablet_manager.cc:1398] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
I20240426 09:49:55.127833 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-8e306113f9874277a60468d9248d45ce received from {username='slave'} at 127.0.99.129:42751
I20240426 09:49:55.127915 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-8e306113f9874277a60468d9248d45ce on tablet 8e306113f9874277a60468d9248d45ce with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:49:55.128557 9397 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:55.128808 9660 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:55.134292 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:55.134438 7255 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:55.136026 6862 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:55.156198 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:55.159240 17878 ts_tablet_manager.cc:902] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:55.159766 17878 tablet_copy_client.cc:308] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:55.160465 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 67e3f93c0a2944e38b7542d1439c907e from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:59473)
I20240426 09:49:55.160571 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 67e3f93c0a2944e38b7542d1439c907e from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:59473: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-67e3f93c0a2944e38b7542d1439c907e
I20240426 09:49:55.161484 9685 tablet_copy_source_session.cc:205] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.161828 18088 tablet.cc:1722] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:55.162140 18088 tablet.cc:1620] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:55.162328 17878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67e3f93c0a2944e38b7542d1439c907e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.164072 17699 tablet.cc:1722] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:55.164244 18089 tablet.cc:1722] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:55.164381 17699 tablet.cc:1620] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:55.164535 18089 tablet.cc:1620] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:55.164882 17878 tablet_copy_client.cc:783] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.165030 17878 tablet_copy_client.cc:647] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.168936 9527 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:55.169025 6991 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:55.169260 17878 tablet_copy_client.cc:523] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.170620 17878 tablet_bootstrap.cc:492] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:55.174050 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=f604704ce5ab44338600852a208d21fa] alter complete (version 2)
I20240426 09:49:55.185545 17878 tablet_bootstrap.cc:492] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.185807 17878 tablet_bootstrap.cc:492] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:49:55.185952 17878 ts_tablet_manager.cc:1367] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.001s
I20240426 09:49:55.186378 17878 raft_consensus.cc:348] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.186477 17878 raft_consensus.cc:729] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.186580 17878 consensus_queue.cc:260] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.186846 17878 ts_tablet_manager.cc:1398] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:55.189301 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-67e3f93c0a2944e38b7542d1439c907e received from {username='slave'} at 127.0.99.130:59473
I20240426 09:49:55.189380 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-67e3f93c0a2944e38b7542d1439c907e on tablet 67e3f93c0a2944e38b7542d1439c907e with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:49:55.208858 17908 ts_tablet_manager.cc:902] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:55.209350 17908 tablet_copy_client.cc:308] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:49:55.209827 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet aebef9b26fec446b8e71d4046768aedc from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:36267)
I20240426 09:49:55.209940 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet aebef9b26fec446b8e71d4046768aedc from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:36267: session id = c244e02f3cf847c0a5b69e1059f1071c-aebef9b26fec446b8e71d4046768aedc
I20240426 09:49:55.211032 9306 tablet_copy_source_session.cc:205] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.212205 17908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aebef9b26fec446b8e71d4046768aedc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.214529 17908 tablet_copy_client.cc:783] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.214707 17908 tablet_copy_client.cc:647] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.223491 6882 raft_consensus.cc:1203] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.227367 7143 raft_consensus.cc:1203] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:55.229110 17908 tablet_copy_client.cc:523] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.231957 17908 tablet_bootstrap.cc:492] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:55.236172 17908 tablet_bootstrap.cc:492] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.236411 17908 tablet_bootstrap.cc:492] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:49:55.236570 17908 ts_tablet_manager.cc:1367] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240426 09:49:55.236940 17908 raft_consensus.cc:348] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.237040 17908 raft_consensus.cc:729] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.237186 17908 consensus_queue.cc:260] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.237426 17908 ts_tablet_manager.cc:1398] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:49:55.238391 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-aebef9b26fec446b8e71d4046768aedc received from {username='slave'} at 127.0.99.129:36267
I20240426 09:49:55.238466 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-aebef9b26fec446b8e71d4046768aedc on tablet aebef9b26fec446b8e71d4046768aedc with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:49:55.249053 18039 ts_tablet_manager.cc:902] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:55.249464 18039 tablet_copy_client.cc:308] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:49:55.250154 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet 536858c15b894b2f82efef38b8f51594 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:53025)
I20240426 09:49:55.250296 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet 536858c15b894b2f82efef38b8f51594 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:53025: session id = e3394a4be9ea4def8aa39b333c82c5ff-536858c15b894b2f82efef38b8f51594
I20240426 09:49:55.251628 9437 tablet_copy_source_session.cc:205] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.253247 18039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 536858c15b894b2f82efef38b8f51594. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.255514 18039 tablet_copy_client.cc:783] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.255667 18039 tablet_copy_client.cc:647] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.258944 18039 tablet_copy_client.cc:523] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.260214 18039 tablet_bootstrap.cc:492] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:55.263720 18039 tablet_bootstrap.cc:492] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.264103 18039 tablet_bootstrap.cc:492] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:49:55.264258 18039 ts_tablet_manager.cc:1367] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:49:55.264663 18039 raft_consensus.cc:348] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:55.264778 18039 raft_consensus.cc:729] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.264947 18039 consensus_queue.cc:260] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:55.265236 18039 ts_tablet_manager.cc:1398] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:55.266633 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-536858c15b894b2f82efef38b8f51594 received from {username='slave'} at 127.0.99.132:53025
I20240426 09:49:55.266736 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-536858c15b894b2f82efef38b8f51594 on tablet 536858c15b894b2f82efef38b8f51594 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:55.276569 17824 ts_tablet_manager.cc:902] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:55.277076 17824 tablet_copy_client.cc:308] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:49:55.277549 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 2770cd77ebb642e3a52183a3871e33fa from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:33703)
I20240426 09:49:55.277657 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 2770cd77ebb642e3a52183a3871e33fa from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:33703: session id = a692602e1a2e4794877e6fb92a7b9962-2770cd77ebb642e3a52183a3871e33fa
I20240426 09:49:55.278995 9685 tablet_copy_source_session.cc:205] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.279433 7011 raft_consensus.cc:1203] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.279888 17824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2770cd77ebb642e3a52183a3871e33fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.282013 17824 tablet_copy_client.cc:783] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.282171 17824 tablet_copy_client.cc:647] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.286114 17824 tablet_copy_client.cc:523] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.287417 17824 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:55.289844 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:55.293112 17824 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.293344 17824 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:49:55.293506 17824 ts_tablet_manager.cc:1367] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20240426 09:49:55.294485 17824 raft_consensus.cc:348] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.294679 17824 raft_consensus.cc:729] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.294891 17824 consensus_queue.cc:260] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.295276 17824 ts_tablet_manager.cc:1398] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:55.295536 18088 tablet.cc:1722] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:55.296113 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-2770cd77ebb642e3a52183a3871e33fa received from {username='slave'} at 127.0.99.133:33703
I20240426 09:49:55.296185 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-2770cd77ebb642e3a52183a3871e33fa on tablet 2770cd77ebb642e3a52183a3871e33fa with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:49:55.296367 18089 tablet.cc:1722] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:55.298082 17699 tablet.cc:1722] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:55.301371 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=f604704ce5ab44338600852a208d21fa] alter complete (version 3)
I20240426 09:49:55.385354 18098 ts_tablet_manager.cc:902] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:55.385831 18098 tablet_copy_client.cc:308] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:49:55.386353 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet 638b15a2c9ea42c3b86a985ccae550a7 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:59663)
I20240426 09:49:55.386466 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet 638b15a2c9ea42c3b86a985ccae550a7 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:59663: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-638b15a2c9ea42c3b86a985ccae550a7
I20240426 09:49:55.387441 9306 tablet_copy_source_session.cc:205] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.388957 18098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 638b15a2c9ea42c3b86a985ccae550a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.392992 18098 tablet_copy_client.cc:783] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.393237 18098 tablet_copy_client.cc:647] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.398206 18210 ts_tablet_manager.cc:902] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:55.399777 18098 tablet_copy_client.cc:523] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.401392 18098 tablet_bootstrap.cc:492] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:55.407199 6740 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:55.407361 6740 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:55.409176 9397 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:55.410099 6740 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 1e2cef7d173846cc895d78cc27f1103e
I20240426 09:49:55.411100 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:55.411145 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2cef7d173846cc895d78cc27f1103e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:55 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:55.411278 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2cef7d173846cc895d78cc27f1103e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:55 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:55.411850 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2cef7d173846cc895d78cc27f1103e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:55 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:55.412137 6991 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:55.417663 18210 tablet_copy_client.cc:308] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:55.418267 18216 ts_tablet_manager.cc:902] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:55.419618 18216 tablet_copy_client.cc:308] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:49:55.419782 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet b2d42c52b7a94e32ab4cdad78ce884ee from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:41659)
I20240426 09:49:55.419883 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet b2d42c52b7a94e32ab4cdad78ce884ee from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:41659: session id = 3b41d87e7bfb419f8aa7500100129b6c-b2d42c52b7a94e32ab4cdad78ce884ee
I20240426 09:49:55.420886 7295 tablet_copy_source_session.cc:205] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.422092 18210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d42c52b7a94e32ab4cdad78ce884ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.424041 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet d83c01e6dc6d4b5b9365243058f459a1 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:41455)
I20240426 09:49:55.424135 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet d83c01e6dc6d4b5b9365243058f459a1 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:41455: session id = 558c4ecd0ece460d9591aa7a3c799d19-d83c01e6dc6d4b5b9365243058f459a1
I20240426 09:49:55.424414 18210 tablet_copy_client.cc:783] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.424578 18210 tablet_copy_client.cc:647] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.424957 7295 tablet_copy_source_session.cc:205] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:49:55.428159 7011 raft_consensus.cc:1203] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.433168 6862 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:55.435766 18214 tablet_replica.cc:330] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:55.436220 18214 raft_consensus.cc:2229] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:55.436728 18214 raft_consensus.cc:2258] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:55.436481 18215 tablet_replica.cc:330] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:55.437088 18215 raft_consensus.cc:2229] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:55.437147 17668 consensus_queue.cc:1035] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.437739 18214 ts_tablet_manager.cc:1875] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:55.438217 18215 raft_consensus.cc:2258] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:55.438876 18215 ts_tablet_manager.cc:1875] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:55.442160 7255 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:55.443173 18098 tablet_bootstrap.cc:492] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.443488 18098 tablet_bootstrap.cc:492] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:49:55.443675 18098 ts_tablet_manager.cc:1367] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.000s
I20240426 09:49:55.449538 18098 raft_consensus.cc:348] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:55.449712 18098 raft_consensus.cc:729] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.449860 18098 consensus_queue.cc:260] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:55.451277 18098 ts_tablet_manager.cc:1398] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:49:55.452639 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-638b15a2c9ea42c3b86a985ccae550a7 received from {username='slave'} at 127.0.99.135:59663
I20240426 09:49:55.452733 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-638b15a2c9ea42c3b86a985ccae550a7 on tablet 638b15a2c9ea42c3b86a985ccae550a7 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:49:55.455137 18213 tablet_replica.cc:330] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:55.456426 18213 raft_consensus.cc:2229] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:55.456646 18213 raft_consensus.cc:2258] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:55.457669 9548 raft_consensus.cc:1203] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.458686 18215 ts_tablet_manager.cc:1888] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:55.459908 18213 ts_tablet_manager.cc:1875] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:55.460646 18214 ts_tablet_manager.cc:1888] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:55.464329 18214 log.cc:1196] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/1e2cef7d173846cc895d78cc27f1103e
I20240426 09:49:55.466102 18214 ts_tablet_manager.cc:1909] T 1e2cef7d173846cc895d78cc27f1103e P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:55.466270 18215 log.cc:1196] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/1e2cef7d173846cc895d78cc27f1103e
I20240426 09:49:55.466503 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:55.467417 18215 ts_tablet_manager.cc:1909] T 1e2cef7d173846cc895d78cc27f1103e P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:55.467391 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 1e2cef7d173846cc895d78cc27f1103e (table rebalancer_dynamic_table_new_name [id=f604704ce5ab44338600852a208d21fa]) successfully deleted
I20240426 09:49:55.469065 18213 ts_tablet_manager.cc:1888] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:55.469254 18213 log.cc:1196] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/1e2cef7d173846cc895d78cc27f1103e
I20240426 09:49:55.469583 18213 ts_tablet_manager.cc:1909] T 1e2cef7d173846cc895d78cc27f1103e P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:55.470932 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 1e2cef7d173846cc895d78cc27f1103e (table rebalancer_dynamic_table_new_name [id=f604704ce5ab44338600852a208d21fa]) successfully deleted
I20240426 09:49:55.471668 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 1e2cef7d173846cc895d78cc27f1103e (table rebalancer_dynamic_table_new_name [id=f604704ce5ab44338600852a208d21fa]) successfully deleted
I20240426 09:49:55.472297 17777 consensus_queue.cc:1035] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.475682 17387 consensus_queue.cc:1035] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.476415 18210 tablet_copy_client.cc:523] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.477957 18210 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:55.479969 7142 raft_consensus.cc:1203] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.481786 18216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83c01e6dc6d4b5b9365243058f459a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.486294 9527 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:55.489506 18216 tablet_copy_client.cc:783] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:49:55.489672 18216 tablet_copy_client.cc:647] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:49:55.495709 18212 consensus_queue.cc:1035] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.496912 9660 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:55.498566 7275 raft_consensus.cc:1203] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.501228 18216 tablet_copy_client.cc:523] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:49:55.502759 18216 tablet_bootstrap.cc:492] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:55.506335 18216 tablet_bootstrap.cc:492] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.506582 18216 tablet_bootstrap.cc:492] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:49:55.506726 18216 ts_tablet_manager.cc:1367] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20240426 09:49:55.507452 18216 raft_consensus.cc:348] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.507575 18216 raft_consensus.cc:729] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.507695 18216 consensus_queue.cc:260] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.507984 18216 ts_tablet_manager.cc:1398] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:55.510788 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-d83c01e6dc6d4b5b9365243058f459a1 received from {username='slave'} at 127.0.99.136:41455
I20240426 09:49:55.510870 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-d83c01e6dc6d4b5b9365243058f459a1 on tablet d83c01e6dc6d4b5b9365243058f459a1 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:55.512544 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:55.521075 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 8f7cbe6f6c2d477e81881c1158adbd12 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=0517f03b84c847178e4b2a291a8f74e6]), partition=
I20240426 09:49:55.521265 9264 tablet_service.cc:1467] Processing CreateTablet for tablet 8f7cbe6f6c2d477e81881c1158adbd12 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=0517f03b84c847178e4b2a291a8f74e6]), partition=
I20240426 09:49:55.521369 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f7cbe6f6c2d477e81881c1158adbd12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.521473 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f7cbe6f6c2d477e81881c1158adbd12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.524111 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 8f7cbe6f6c2d477e81881c1158adbd12 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=0517f03b84c847178e4b2a291a8f74e6]), partition=
I20240426 09:49:55.524394 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f7cbe6f6c2d477e81881c1158adbd12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.529191 18233 tablet_bootstrap.cc:492] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:55.529431 18234 tablet_bootstrap.cc:492] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:55.530442 18234 tablet_bootstrap.cc:654] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.531450 18233 tablet_bootstrap.cc:654] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.532181 18235 tablet_bootstrap.cc:492] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:55.538543 18210 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:49:55.538875 18210 tablet_bootstrap.cc:492] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:49:55.539104 18210 ts_tablet_manager.cc:1367] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.061s user 0.027s sys 0.001s
I20240426 09:49:55.539552 18210 raft_consensus.cc:348] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:55.539680 18210 raft_consensus.cc:729] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.539816 18210 consensus_queue.cc:260] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:55.541764 18234 tablet_bootstrap.cc:492] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:55.541913 18234 ts_tablet_manager.cc:1367] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:55.542629 18234 raft_consensus.cc:348] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.543694 18234 raft_consensus.cc:374] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.543797 18234 raft_consensus.cc:729] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.543793 18212 consensus_queue.cc:1035] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.544065 18234 consensus_queue.cc:260] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.544603 18210 ts_tablet_manager.cc:1398] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240426 09:49:55.545300 18234 ts_tablet_manager.cc:1398] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:55.546128 18233 tablet_bootstrap.cc:492] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:55.546272 18233 ts_tablet_manager.cc:1367] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240426 09:49:55.546790 18233 raft_consensus.cc:348] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.546969 18233 raft_consensus.cc:374] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.547063 18233 raft_consensus.cc:729] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.547276 18233 consensus_queue.cc:260] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.547646 18233 ts_tablet_manager.cc:1398] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:55.550900 18235 tablet_bootstrap.cc:654] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.556985 18235 tablet_bootstrap.cc:492] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:55.557109 18235 ts_tablet_manager.cc:1367] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.002s
I20240426 09:49:55.557740 18235 raft_consensus.cc:348] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.557880 18235 raft_consensus.cc:374] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.557922 18235 raft_consensus.cc:729] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.558066 18235 consensus_queue.cc:260] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.558475 18235 ts_tablet_manager.cc:1398] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:55.563270 7142 raft_consensus.cc:1203] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.567853 9285 raft_consensus.cc:1203] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.570044 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-b2d42c52b7a94e32ab4cdad78ce884ee received from {username='slave'} at 127.0.99.134:41659
I20240426 09:49:55.570133 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-b2d42c52b7a94e32ab4cdad78ce884ee on tablet b2d42c52b7a94e32ab4cdad78ce884ee with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:49:55.580138 17755 consensus_queue.cc:1035] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.586359 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:55.609675 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 14752ab4ee9c4389b9d7afbfe1f5534c (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:55.609989 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14752ab4ee9c4389b9d7afbfe1f5534c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.609994 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 14752ab4ee9c4389b9d7afbfe1f5534c (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:55.610319 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14752ab4ee9c4389b9d7afbfe1f5534c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.611127 9661 tablet_service.cc:1467] Processing CreateTablet for tablet ab631877cfb646829d984aacd8a02795 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:55.611397 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab631877cfb646829d984aacd8a02795. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.614207 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 14752ab4ee9c4389b9d7afbfe1f5534c (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:55.614543 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14752ab4ee9c4389b9d7afbfe1f5534c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.617051 18233 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:55.618031 18233 tablet_bootstrap.cc:654] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.618060 6882 raft_consensus.cc:1203] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:49:55.623387 9264 tablet_service.cc:1467] Processing CreateTablet for tablet ab631877cfb646829d984aacd8a02795 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:55.623737 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab631877cfb646829d984aacd8a02795. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.625325 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2f793f7e7a4ff596379c1cfe9d251b (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:55.625586 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2f793f7e7a4ff596379c1cfe9d251b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.626397 18234 tablet_bootstrap.cc:492] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:55.627009 18245 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:55.628149 18234 tablet_bootstrap.cc:654] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.628436 6991 tablet_service.cc:1467] Processing CreateTablet for tablet ab631877cfb646829d984aacd8a02795 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:55.628686 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab631877cfb646829d984aacd8a02795. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.628448 6992 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2f793f7e7a4ff596379c1cfe9d251b (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:55.629215 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2f793f7e7a4ff596379c1cfe9d251b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.633195 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 6e2f793f7e7a4ff596379c1cfe9d251b (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:55.634086 6882 raft_consensus.cc:1203] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.634416 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e2f793f7e7a4ff596379c1cfe9d251b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:55.627928 18245 tablet_bootstrap.cc:654] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.642398 7011 raft_consensus.cc:1203] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.647640 18245 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:55.647821 18245 ts_tablet_manager.cc:1367] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240426 09:49:55.649430 18245 raft_consensus.cc:348] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.649552 18245 raft_consensus.cc:374] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.649679 18245 raft_consensus.cc:729] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.649852 18245 consensus_queue.cc:260] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.651026 18246 tablet_bootstrap.cc:492] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:55.651072 18233 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:55.651070 18245 ts_tablet_manager.cc:1398] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:55.651165 18233 ts_tablet_manager.cc:1367] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20240426 09:49:55.651232 18245 tablet_bootstrap.cc:492] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:55.651335 18248 tablet_bootstrap.cc:492] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:55.651621 18233 raft_consensus.cc:348] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.651754 18233 raft_consensus.cc:374] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.651795 18233 raft_consensus.cc:729] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.651913 18233 consensus_queue.cc:260] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.652179 18233 ts_tablet_manager.cc:1398] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:55.652339 18233 tablet_bootstrap.cc:492] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:55.652438 18245 tablet_bootstrap.cc:654] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.652799 18246 tablet_bootstrap.cc:654] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.653214 18233 tablet_bootstrap.cc:654] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.654675 18248 tablet_bootstrap.cc:654] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.659564 18234 tablet_bootstrap.cc:492] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:55.659669 18234 ts_tablet_manager.cc:1367] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.000s
I20240426 09:49:55.659718 18246 tablet_bootstrap.cc:492] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:55.659803 18246 ts_tablet_manager.cc:1367] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:55.660184 18234 raft_consensus.cc:348] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.660305 18234 raft_consensus.cc:374] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.660337 18234 raft_consensus.cc:729] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.660454 18234 consensus_queue.cc:260] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.660741 18234 ts_tablet_manager.cc:1398] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:55.660225 18246 raft_consensus.cc:348] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.661360 18246 raft_consensus.cc:374] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.661401 18246 raft_consensus.cc:729] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.661518 18246 consensus_queue.cc:260] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.663880 18246 ts_tablet_manager.cc:1398] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:55.664175 18246 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:55.666647 18246 tablet_bootstrap.cc:654] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.666867 18245 tablet_bootstrap.cc:492] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:55.667032 18245 ts_tablet_manager.cc:1367] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240426 09:49:55.667541 18245 raft_consensus.cc:348] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.667717 18245 raft_consensus.cc:374] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.667793 18245 raft_consensus.cc:729] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.667974 18245 consensus_queue.cc:260] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.669126 18245 ts_tablet_manager.cc:1398] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:55.669616 18245 tablet_bootstrap.cc:492] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:55.671057 18248 tablet_bootstrap.cc:492] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:55.671227 18248 ts_tablet_manager.cc:1367] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.001s
I20240426 09:49:55.672305 18233 tablet_bootstrap.cc:492] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:55.672412 18233 ts_tablet_manager.cc:1367] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20240426 09:49:55.672556 18248 raft_consensus.cc:348] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.672698 18248 raft_consensus.cc:374] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.672736 18248 raft_consensus.cc:729] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.672878 18248 consensus_queue.cc:260] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.673024 18233 raft_consensus.cc:348] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.673121 18233 raft_consensus.cc:374] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.673149 18233 raft_consensus.cc:729] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.673234 18248 ts_tablet_manager.cc:1398] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240426 09:49:55.673251 18233 consensus_queue.cc:260] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.673498 18233 ts_tablet_manager.cc:1398] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:55.674733 18246 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:55.674856 18246 ts_tablet_manager.cc:1367] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:55.675421 18246 raft_consensus.cc:348] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.675539 18246 raft_consensus.cc:374] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.675575 18246 raft_consensus.cc:729] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.675771 18246 consensus_queue.cc:260] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.677138 7275 raft_consensus.cc:1203] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.677400 18246 ts_tablet_manager.cc:1398] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:55.683007 17690 raft_consensus.cc:482] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:55.683094 17690 raft_consensus.cc:504] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.683444 17690 leader_election.cc:288] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:55.683887 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14752ab4ee9c4389b9d7afbfe1f5534c" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:55.683941 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14752ab4ee9c4389b9d7afbfe1f5534c" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:55.683988 9669 raft_consensus.cc:2454] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:55.684041 9417 raft_consensus.cc:2454] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:55.684279 6946 leader_election.cc:302] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:55.684943 17690 raft_consensus.cc:2781] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:55.684993 17690 raft_consensus.cc:482] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:55.685029 17690 raft_consensus.cc:3037] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.686089 17690 raft_consensus.cc:504] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.686301 18245 tablet_bootstrap.cc:654] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:55.686410 17690 leader_election.cc:288] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:55.690042 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14752ab4ee9c4389b9d7afbfe1f5534c" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:55.690142 9669 raft_consensus.cc:3037] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.690217 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14752ab4ee9c4389b9d7afbfe1f5534c" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:55.690310 9417 raft_consensus.cc:3037] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.691148 9669 raft_consensus.cc:2454] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:55.691300 9417 raft_consensus.cc:2454] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:55.693022 6946 leader_election.cc:302] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:55.693298 17690 raft_consensus.cc:2781] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:55.693435 17690 raft_consensus.cc:686] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:55.693591 17690 consensus_queue.cc:237] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:55.693863 17668 raft_consensus.cc:482] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:55.693934 17668 raft_consensus.cc:504] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.694262 17668 leader_election.cc:288] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:55.694540 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab631877cfb646829d984aacd8a02795" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:55.694676 7011 raft_consensus.cc:2454] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:55.695003 9614 leader_election.cc:302] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:55.696122 17668 raft_consensus.cc:2781] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:55.696177 17668 raft_consensus.cc:482] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:55.696213 17668 raft_consensus.cc:3037] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.696678 6738 catalog_manager.cc:5484] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:55.697183 17668 raft_consensus.cc:504] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.697369 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab631877cfb646829d984aacd8a02795" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:55.697479 9286 raft_consensus.cc:2454] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:55.697504 17668 leader_election.cc:288] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:55.698348 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab631877cfb646829d984aacd8a02795" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:55.698451 9286 raft_consensus.cc:3037] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.698632 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab631877cfb646829d984aacd8a02795" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:55.698710 7011 raft_consensus.cc:3037] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.699431 9286 raft_consensus.cc:2454] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:55.699610 7011 raft_consensus.cc:2454] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:55.699754 9615 leader_election.cc:302] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:55.700165 17668 raft_consensus.cc:2781] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:55.700243 17668 raft_consensus.cc:686] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:55.700349 17668 consensus_queue.cc:237] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:55.701069 18245 tablet_bootstrap.cc:492] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:55.701141 18245 ts_tablet_manager.cc:1367] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.000s
I20240426 09:49:55.701565 18245 raft_consensus.cc:348] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.701665 18245 raft_consensus.cc:374] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:55.701694 18245 raft_consensus.cc:729] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:55.701814 18245 consensus_queue.cc:260] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.702085 18245 ts_tablet_manager.cc:1398] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:55.703203 6740 catalog_manager.cc:5484] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:55.726478 17905 raft_consensus.cc:482] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:55.726601 17905 raft_consensus.cc:504] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.727165 17905 leader_election.cc:288] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:55.727849 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7cbe6f6c2d477e81881c1158adbd12" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:55.727974 9286 raft_consensus.cc:2454] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:55.728274 9351 leader_election.cc:302] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:55.728518 17905 raft_consensus.cc:2781] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:55.728600 17905 raft_consensus.cc:482] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:55.728646 17905 raft_consensus.cc:3037] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.730103 17905 raft_consensus.cc:504] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.730589 17905 leader_election.cc:288] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:55.730574 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7cbe6f6c2d477e81881c1158adbd12" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:55.730698 7143 raft_consensus.cc:2454] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:55.731341 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7cbe6f6c2d477e81881c1158adbd12" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:55.731436 7143 raft_consensus.cc:3037] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.731386 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7cbe6f6c2d477e81881c1158adbd12" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:55.731549 9286 raft_consensus.cc:3037] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.732352 7143 raft_consensus.cc:2454] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:55.732802 9286 raft_consensus.cc:2454] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:55.732879 9350 leader_election.cc:302] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:55.733165 17905 raft_consensus.cc:2781] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:55.733268 17905 raft_consensus.cc:686] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:55.733392 17905 consensus_queue.cc:237] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:55.735826 6740 catalog_manager.cc:5484] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:55.767469 9548 raft_consensus.cc:1203] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.826834 17690 raft_consensus.cc:482] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:55.826957 17690 raft_consensus.cc:504] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.827315 17690 leader_election.cc:288] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:55.828963 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f793f7e7a4ff596379c1cfe9d251b" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:55.829092 9417 raft_consensus.cc:2454] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:55.829757 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f793f7e7a4ff596379c1cfe9d251b" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:55.829875 6882 raft_consensus.cc:2454] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:55.830250 6948 leader_election.cc:302] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:55.831068 18261 raft_consensus.cc:2781] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:55.831126 18261 raft_consensus.cc:482] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:55.831176 18261 raft_consensus.cc:3037] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.832332 18261 raft_consensus.cc:504] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.832659 18261 leader_election.cc:288] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:55.833204 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f793f7e7a4ff596379c1cfe9d251b" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:55.833308 6882 raft_consensus.cc:3037] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.834306 6882 raft_consensus.cc:2454] T 6e2f793f7e7a4ff596379c1cfe9d251b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:55.834650 6948 leader_election.cc:302] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:55.835000 16849 raft_consensus.cc:2781] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:55.835063 16849 raft_consensus.cc:686] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:55.835181 16849 consensus_queue.cc:237] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:55.836895 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e2f793f7e7a4ff596379c1cfe9d251b" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:55.837005 9417 raft_consensus.cc:3037] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:55.837936 9417 raft_consensus.cc:2454] T 6e2f793f7e7a4ff596379c1cfe9d251b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:55.839733 6731 catalog_manager.cc:5484] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:55.844974 9286 raft_consensus.cc:1203] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.852008 6731 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:55.860441 9286 raft_consensus.cc:1261] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:55.860973 7143 raft_consensus.cc:1261] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:55.861282 17905 consensus_queue.cc:1035] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.864696 17905 consensus_queue.cc:1035] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:55.864957 6731 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet 8f7cbe6f6c2d477e81881c1158adbd12 (table rebalancer_dynamic_table [id=0517f03b84c847178e4b2a291a8f74e6]). Expected version 1 got 0
I20240426 09:49:55.869819 6731 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet 8f7cbe6f6c2d477e81881c1158adbd12 (table rebalancer_dynamic_table [id=0517f03b84c847178e4b2a291a8f74e6]). Expected version 1 got 0
I20240426 09:49:55.871903 18308 tablet.cc:1722] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:55.872292 18308 tablet.cc:1620] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:55.873531 6734 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet 8f7cbe6f6c2d477e81881c1158adbd12 (table rebalancer_dynamic_table [id=0517f03b84c847178e4b2a291a8f74e6]). Expected version 1 got 0
I20240426 09:49:55.876819 6740 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet 8f7cbe6f6c2d477e81881c1158adbd12 (table rebalancer_dynamic_table [id=0517f03b84c847178e4b2a291a8f74e6]). Expected version 1 got 0
I20240426 09:49:55.876897 18306 tablet.cc:1722] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:55.877424 18306 tablet.cc:1620] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:55.882305 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=0517f03b84c847178e4b2a291a8f74e6] alter complete (version 1)
I20240426 09:49:55.883541 18307 tablet.cc:1722] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:55.884119 18307 tablet.cc:1620] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:55.912990 6992 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:49:55.914336 9669 raft_consensus.cc:1203] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:49:55.921764 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:55.922909 7255 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:55.926914 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:55.935590 9397 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:55.964917 9660 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:55.970827 9527 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:55.986089 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:55.989490 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:55.994988 9416 raft_consensus.cc:1203] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240426 09:49:56.000409 18306 tablet.cc:1722] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:56.000741 18306 tablet.cc:1620] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:56.001025 18308 tablet.cc:1722] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:56.001307 18308 tablet.cc:1620] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:56.002717 18307 tablet.cc:1722] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:56.003053 18307 tablet.cc:1620] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:56.007577 6740 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=0517f03b84c847178e4b2a291a8f74e6] alter complete (version 2)
I20240426 09:49:56.050509 9416 consensus_queue.cc:237] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:56.051735 7275 raft_consensus.cc:1261] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:56.051829 9285 raft_consensus.cc:1261] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:56.052161 17511 consensus_queue.cc:1035] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:56.052296 17905 consensus_queue.cc:1035] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:56.054327 17511 raft_consensus.cc:2932] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:56.054636 7275 raft_consensus.cc:2932] T 536858c15b894b2f82efef38b8f51594 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:56.055979 9285 raft_consensus.cc:2932] T 536858c15b894b2f82efef38b8f51594 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:56.057686 6731 catalog_manager.cc:5484] T 536858c15b894b2f82efef38b8f51594 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:56.060690 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet 536858c15b894b2f82efef38b8f51594 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47654
I20240426 09:49:56.061053 18340 tablet_replica.cc:330] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:49:56.061297 18340 raft_consensus.cc:2229] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:56.061543 18340 raft_consensus.cc:2258] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:56.062084 18340 ts_tablet_manager.cc:1875] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:56.064240 18340 ts_tablet_manager.cc:1888] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:56.064309 18340 log.cc:1196] T 536858c15b894b2f82efef38b8f51594 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/536858c15b894b2f82efef38b8f51594
I20240426 09:49:56.064954 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 536858c15b894b2f82efef38b8f51594 (table rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]) successfully deleted
I20240426 09:49:56.083436 17668 consensus_queue.cc:1035] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.086156 17777 consensus_queue.cc:1035] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.111001 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:56.117511 18306 tablet.cc:1722] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:56.118052 18308 tablet.cc:1722] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:56.119174 18307 tablet.cc:1722] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:56.123672 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=0517f03b84c847178e4b2a291a8f74e6] alter complete (version 3)
I20240426 09:49:56.178140 18261 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.185092 16849 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.219615 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:56.221889 6992 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:49:56.223219 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:56.224309 9660 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:56.231567 6733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:56.231730 6733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:56.235888 6733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 8f7cbe6f6c2d477e81881c1158adbd12
I20240426 09:49:56.236459 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f7cbe6f6c2d477e81881c1158adbd12 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:56 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:56.236716 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f7cbe6f6c2d477e81881c1158adbd12 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:56 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:56.236765 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f7cbe6f6c2d477e81881c1158adbd12 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:56 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:56.237007 18400 tablet_replica.cc:330] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:56.237188 18400 raft_consensus.cc:2229] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:56.237432 18400 raft_consensus.cc:2258] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:56.237563 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:56.238124 18400 ts_tablet_manager.cc:1875] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:56.240839 18400 ts_tablet_manager.cc:1888] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:56.243638 18401 tablet_replica.cc:330] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:56.243813 18399 tablet_replica.cc:330] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:56.244004 18399 raft_consensus.cc:2229] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:56.244195 18401 raft_consensus.cc:2229] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:56.244369 18399 raft_consensus.cc:2258] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:56.244644 18401 raft_consensus.cc:2258] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:56.245419 18399 ts_tablet_manager.cc:1875] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:56.245826 18401 ts_tablet_manager.cc:1875] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:56.251119 18399 ts_tablet_manager.cc:1888] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:56.251377 18401 ts_tablet_manager.cc:1888] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:56.251495 18401 log.cc:1196] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/8f7cbe6f6c2d477e81881c1158adbd12
I20240426 09:49:56.251808 18401 ts_tablet_manager.cc:1909] T 8f7cbe6f6c2d477e81881c1158adbd12 P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:56.251929 18399 log.cc:1196] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/8f7cbe6f6c2d477e81881c1158adbd12
I20240426 09:49:56.252310 18399 ts_tablet_manager.cc:1909] T 8f7cbe6f6c2d477e81881c1158adbd12 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:56.246326 9397 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240426 09:49:56.252877 18400 log.cc:1196] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/8f7cbe6f6c2d477e81881c1158adbd12
I20240426 09:49:56.253294 18400 ts_tablet_manager.cc:1909] T 8f7cbe6f6c2d477e81881c1158adbd12 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:56.253515 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 8f7cbe6f6c2d477e81881c1158adbd12 (table rebalancer_dynamic_table_new_name [id=0517f03b84c847178e4b2a291a8f74e6]) successfully deleted
I20240426 09:49:56.253880 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 8f7cbe6f6c2d477e81881c1158adbd12 (table rebalancer_dynamic_table_new_name [id=0517f03b84c847178e4b2a291a8f74e6]) successfully deleted
I20240426 09:49:56.254081 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 8f7cbe6f6c2d477e81881c1158adbd12 (table rebalancer_dynamic_table_new_name [id=0517f03b84c847178e4b2a291a8f74e6]) successfully deleted
I20240426 09:49:56.262420 17690 consensus_queue.cc:1035] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.268370 7255 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:56.268620 18261 consensus_queue.cc:1035] T 6e2f793f7e7a4ff596379c1cfe9d251b P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.270906 9527 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:56.310561 6882 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0d04a7ec2c62437b879de204b5c885e9"
dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:57208
I20240426 09:49:56.310696 6882 raft_consensus.cc:595] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Received request to transfer leadership
I20240426 09:49:56.318195 18240 raft_consensus.cc:979] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: : Instructing follower b8e863bc0edf4940bbf3e9f9b2bbb3e5 to start an election
I20240426 09:49:56.318306 18240 raft_consensus.cc:1067] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Signalling peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 to start an election
I20240426 09:49:56.318723 9548 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0d04a7ec2c62437b879de204b5c885e9"
dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
from {username='slave'} at 127.0.99.129:35867
I20240426 09:49:56.318897 9548 raft_consensus.cc:482] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:49:56.318950 9548 raft_consensus.cc:3037] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:56.319985 9548 raft_consensus.cc:504] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.320503 9548 leader_election.cc:288] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 2 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:56.320986 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d04a7ec2c62437b879de204b5c885e9" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:56.321115 7275 raft_consensus.cc:3037] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:56.321086 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d04a7ec2c62437b879de204b5c885e9" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:56.321141 6882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d04a7ec2c62437b879de204b5c885e9" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:56.321198 7011 raft_consensus.cc:3037] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:56.321247 6882 raft_consensus.cc:3032] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:49:56.321280 6882 raft_consensus.cc:729] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:49:56.321400 6882 consensus_queue.cc:260] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.321576 6882 raft_consensus.cc:3037] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:49:56.322134 7275 raft_consensus.cc:2454] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 2.
I20240426 09:49:56.322134 7011 raft_consensus.cc:2454] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 2.
I20240426 09:49:56.322546 6882 raft_consensus.cc:2454] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 2.
I20240426 09:49:56.322950 9482 leader_election.cc:302] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:56.323166 17387 raft_consensus.cc:2781] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:49:56.323274 17387 raft_consensus.cc:686] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:56.323417 17387 consensus_queue.cc:237] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.326416 6733 catalog_manager.cc:5484] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 1 to 2, leader changed from c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 2 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:56.337505 6733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:56.344094 9264 tablet_service.cc:1467] Processing CreateTablet for tablet a1e3e872ead24205ab597843fd8e167f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=54beaf465fd943cfb6f361d5a6c661f1]), partition=
I20240426 09:49:56.344362 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e3e872ead24205ab597843fd8e167f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.346706 18422 tablet_bootstrap.cc:492] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:56.347054 6862 tablet_service.cc:1467] Processing CreateTablet for tablet a1e3e872ead24205ab597843fd8e167f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=54beaf465fd943cfb6f361d5a6c661f1]), partition=
I20240426 09:49:56.347272 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e3e872ead24205ab597843fd8e167f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.347783 18422 tablet_bootstrap.cc:654] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.349386 18427 tablet_bootstrap.cc:492] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:56.350370 7123 tablet_service.cc:1467] Processing CreateTablet for tablet a1e3e872ead24205ab597843fd8e167f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=54beaf465fd943cfb6f361d5a6c661f1]), partition=
I20240426 09:49:56.350436 18427 tablet_bootstrap.cc:654] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.350585 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e3e872ead24205ab597843fd8e167f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.350847 18422 tablet_bootstrap.cc:492] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:56.350934 18422 ts_tablet_manager.cc:1367] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:56.351449 18427 tablet_bootstrap.cc:492] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:56.351521 18427 ts_tablet_manager.cc:1367] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:56.351426 18422 raft_consensus.cc:348] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:56.351562 18422 raft_consensus.cc:374] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.351603 18422 raft_consensus.cc:729] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.351720 18422 consensus_queue.cc:260] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:56.351929 18427 raft_consensus.cc:348] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:56.351995 18422 ts_tablet_manager.cc:1398] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.352026 18427 raft_consensus.cc:374] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.352057 18427 raft_consensus.cc:729] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.352169 18427 consensus_queue.cc:260] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:56.352458 18427 ts_tablet_manager.cc:1398] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.354534 18430 tablet_bootstrap.cc:492] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:56.357867 18430 tablet_bootstrap.cc:654] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.358726 6733 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:56.360028 18430 tablet_bootstrap.cc:492] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:56.360108 18430 ts_tablet_manager.cc:1367] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:56.360534 18430 raft_consensus.cc:348] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:56.360627 18430 raft_consensus.cc:374] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.360656 18430 raft_consensus.cc:729] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.360762 18430 consensus_queue.cc:260] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:56.361003 18430 ts_tablet_manager.cc:1398] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.367847 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 3086b0bb58174f4db7c29db9a74dae88 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:56.368163 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3086b0bb58174f4db7c29db9a74dae88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.368204 9395 tablet_service.cc:1467] Processing CreateTablet for tablet 158666afa8f14791addcd6be64e6c562 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:56.369016 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 01d9726ca8b641178824a7fa5977bbdf (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:56.369225 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d9726ca8b641178824a7fa5977bbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.370446 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 3086b0bb58174f4db7c29db9a74dae88 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:56.370569 18433 tablet_bootstrap.cc:492] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:56.370887 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 01d9726ca8b641178824a7fa5977bbdf (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:56.371305 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 158666afa8f14791addcd6be64e6c562 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:56.371515 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158666afa8f14791addcd6be64e6c562. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.371497 18434 tablet_bootstrap.cc:492] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:56.372223 9395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158666afa8f14791addcd6be64e6c562. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.372601 18434 tablet_bootstrap.cc:654] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.373553 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 158666afa8f14791addcd6be64e6c562 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:56.373812 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158666afa8f14791addcd6be64e6c562. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.374373 18436 tablet_bootstrap.cc:492] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:56.375181 18434 tablet_bootstrap.cc:492] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:56.375263 18434 ts_tablet_manager.cc:1367] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:56.375739 18434 raft_consensus.cc:348] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.375854 18434 raft_consensus.cc:374] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.375890 18434 raft_consensus.cc:729] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.376008 9264 tablet_service.cc:1467] Processing CreateTablet for tablet 3086b0bb58174f4db7c29db9a74dae88 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:56.376036 18434 consensus_queue.cc:260] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.376240 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3086b0bb58174f4db7c29db9a74dae88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.376372 18434 ts_tablet_manager.cc:1398] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.376869 18436 tablet_bootstrap.cc:654] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.377770 18436 tablet_bootstrap.cc:492] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:56.377830 18436 ts_tablet_manager.cc:1367] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:56.378190 18422 tablet_bootstrap.cc:492] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:56.378223 18436 raft_consensus.cc:348] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.378307 18436 raft_consensus.cc:374] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.378394 18436 raft_consensus.cc:729] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.378532 18436 consensus_queue.cc:260] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.378857 18436 ts_tablet_manager.cc:1398] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.379189 18422 tablet_bootstrap.cc:654] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.379583 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d9726ca8b641178824a7fa5977bbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.382494 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3086b0bb58174f4db7c29db9a74dae88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.383499 18422 tablet_bootstrap.cc:492] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:56.383596 18422 ts_tablet_manager.cc:1367] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:56.383977 18422 raft_consensus.cc:348] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.384089 18422 raft_consensus.cc:374] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.384126 18422 raft_consensus.cc:729] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.384434 18436 tablet_bootstrap.cc:492] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:56.385421 18436 tablet_bootstrap.cc:654] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.385380 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 01d9726ca8b641178824a7fa5977bbdf (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:56.385666 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d9726ca8b641178824a7fa5977bbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:56.386412 18436 tablet_bootstrap.cc:492] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:56.386435 18433 tablet_bootstrap.cc:654] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.386492 18436 ts_tablet_manager.cc:1367] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:56.386624 18422 consensus_queue.cc:260] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.387008 18436 raft_consensus.cc:348] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.387097 18436 raft_consensus.cc:374] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.387125 18436 raft_consensus.cc:729] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.387229 18436 consensus_queue.cc:260] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.387486 18436 ts_tablet_manager.cc:1398] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.388085 18422 ts_tablet_manager.cc:1398] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240426 09:49:56.388945 18427 tablet_bootstrap.cc:492] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:56.389860 18427 tablet_bootstrap.cc:654] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.394352 18430 tablet_bootstrap.cc:492] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:56.395545 18430 tablet_bootstrap.cc:654] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.396472 18436 tablet_bootstrap.cc:492] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:56.397467 18436 tablet_bootstrap.cc:654] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.398233 18430 tablet_bootstrap.cc:492] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:56.398350 18430 ts_tablet_manager.cc:1367] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:56.398864 18430 raft_consensus.cc:348] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.398983 18430 raft_consensus.cc:374] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.399013 18430 raft_consensus.cc:729] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.399147 18430 consensus_queue.cc:260] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.399353 18433 tablet_bootstrap.cc:492] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:56.399432 18430 ts_tablet_manager.cc:1398] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.399433 18433 ts_tablet_manager.cc:1367] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20240426 09:49:56.399958 18436 tablet_bootstrap.cc:492] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:56.400041 18436 ts_tablet_manager.cc:1367] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:56.400199 18433 raft_consensus.cc:348] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.400298 18433 raft_consensus.cc:374] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.400331 18433 raft_consensus.cc:729] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.400504 18436 raft_consensus.cc:348] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.400660 18436 raft_consensus.cc:374] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.400465 18433 consensus_queue.cc:260] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.400715 18436 raft_consensus.cc:729] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.401144 18436 consensus_queue.cc:260] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.401234 18427 tablet_bootstrap.cc:492] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:56.401309 18427 ts_tablet_manager.cc:1367] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:49:56.401409 18436 ts_tablet_manager.cc:1398] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.401763 18427 raft_consensus.cc:348] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.401877 18427 raft_consensus.cc:374] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.401911 18427 raft_consensus.cc:729] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.402014 18427 consensus_queue.cc:260] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.402276 18427 ts_tablet_manager.cc:1398] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.402315 17905 raft_consensus.cc:482] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:56.402375 17905 raft_consensus.cc:504] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.402812 17905 leader_election.cc:288] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:56.403764 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3086b0bb58174f4db7c29db9a74dae88" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:56.403867 9669 raft_consensus.cc:2454] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:56.404233 9350 leader_election.cc:302] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:56.404687 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3086b0bb58174f4db7c29db9a74dae88" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:56.404804 9285 raft_consensus.cc:2454] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:56.405769 17905 raft_consensus.cc:2781] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:56.405848 17905 raft_consensus.cc:482] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:56.405894 17905 raft_consensus.cc:3037] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.407013 17905 raft_consensus.cc:504] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.407325 17905 leader_election.cc:288] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:56.407786 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3086b0bb58174f4db7c29db9a74dae88" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:56.407903 9285 raft_consensus.cc:3037] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.408826 9285 raft_consensus.cc:2454] T 3086b0bb58174f4db7c29db9a74dae88 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:56.409363 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3086b0bb58174f4db7c29db9a74dae88" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:56.409466 9669 raft_consensus.cc:3037] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.410391 9669 raft_consensus.cc:2454] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:56.410928 9351 leader_election.cc:302] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:56.411327 18339 raft_consensus.cc:2781] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:56.411424 18339 raft_consensus.cc:686] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:56.411581 18339 consensus_queue.cc:237] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.414222 18433 ts_tablet_manager.cc:1398] T 3086b0bb58174f4db7c29db9a74dae88 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.014s user 0.001s sys 0.000s
I20240426 09:49:56.414371 17461 raft_consensus.cc:482] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:56.414464 17461 raft_consensus.cc:504] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:56.414851 17461 leader_election.cc:288] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:56.415081 17511 raft_consensus.cc:482] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:56.415174 17511 raft_consensus.cc:504] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.415410 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e3e872ead24205ab597843fd8e167f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:56.415529 17511 leader_election.cc:288] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:56.415530 6880 raft_consensus.cc:2454] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:56.415793 7079 leader_election.cc:302] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:56.416200 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158666afa8f14791addcd6be64e6c562" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:56.416307 7142 raft_consensus.cc:2454] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:56.416527 18133 raft_consensus.cc:2781] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:56.416599 18133 raft_consensus.cc:482] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:56.416643 18133 raft_consensus.cc:3037] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.416760 9350 leader_election.cc:302] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:56.417344 17511 raft_consensus.cc:2781] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:56.417398 17511 raft_consensus.cc:482] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:56.417433 17511 raft_consensus.cc:3037] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.417636 18133 raft_consensus.cc:504] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:56.417662 6736 catalog_manager.cc:5484] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:56.417961 18133 leader_election.cc:288] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:56.418233 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e3e872ead24205ab597843fd8e167f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:56.418341 9286 raft_consensus.cc:3037] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.418367 17511 raft_consensus.cc:504] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.418486 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e3e872ead24205ab597843fd8e167f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:56.418581 6880 raft_consensus.cc:3037] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.418663 17511 leader_election.cc:288] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:56.419351 9286 raft_consensus.cc:2454] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:56.419536 6880 raft_consensus.cc:2454] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:56.419804 7078 leader_election.cc:302] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:56.420106 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158666afa8f14791addcd6be64e6c562" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:56.420217 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158666afa8f14791addcd6be64e6c562" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:56.420312 7142 raft_consensus.cc:3037] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
W20240426 09:49:56.421005 9350 leader_election.cc:341] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:56.421142 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "158666afa8f14791addcd6be64e6c562" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:56.421263 7142 raft_consensus.cc:2454] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
W20240426 09:49:56.421430 9350 leader_election.cc:341] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:56.421615 18133 raft_consensus.cc:2781] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:56.421660 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e3e872ead24205ab597843fd8e167f" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:56.421710 18133 raft_consensus.cc:686] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:56.421749 9285 raft_consensus.cc:2362] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1. Re-sending same reply.
I20240426 09:49:56.421826 18133 consensus_queue.cc:237] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:56.422129 9350 leader_election.cc:302] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 3b41d87e7bfb419f8aa7500100129b6c; no voters: 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:49:56.422267 17511 raft_consensus.cc:2781] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:56.422348 17511 raft_consensus.cc:686] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:56.422475 17511 consensus_queue.cc:237] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.424338 6738 catalog_manager.cc:5484] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:56.426928 18433 tablet_bootstrap.cc:492] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:56.430542 6732 catalog_manager.cc:5484] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:56.432433 18433 tablet_bootstrap.cc:654] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:56.438377 18433 tablet_bootstrap.cc:492] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:56.438479 18433 ts_tablet_manager.cc:1367] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20240426 09:49:56.438951 18433 raft_consensus.cc:348] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.439133 18433 raft_consensus.cc:374] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:56.439190 18433 raft_consensus.cc:729] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:56.439316 18433 consensus_queue.cc:260] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:56.439786 18433 ts_tablet_manager.cc:1398] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:56.531703 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:56.535827 9285 raft_consensus.cc:1261] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:56.535944 6880 raft_consensus.cc:1261] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:56.536375 17461 consensus_queue.cc:1035] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.536509 18133 consensus_queue.cc:1035] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.541044 18307 tablet.cc:1722] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:56.541026 18308 tablet.cc:1722] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:56.541098 6731 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet a1e3e872ead24205ab597843fd8e167f (table rebalancer_dynamic_table [id=54beaf465fd943cfb6f361d5a6c661f1]). Expected version 1 got 0
I20240426 09:49:56.541332 18308 tablet.cc:1620] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:56.541337 18307 tablet.cc:1620] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:56.541452 6732 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet a1e3e872ead24205ab597843fd8e167f (table rebalancer_dynamic_table [id=54beaf465fd943cfb6f361d5a6c661f1]). Expected version 1 got 0
I20240426 09:49:56.543967 6732 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet a1e3e872ead24205ab597843fd8e167f (table rebalancer_dynamic_table [id=54beaf465fd943cfb6f361d5a6c661f1]). Expected version 1 got 0
I20240426 09:49:56.546259 18471 tablet.cc:1722] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:56.546630 18471 tablet.cc:1620] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:56.549561 6732 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=54beaf465fd943cfb6f361d5a6c661f1] alter complete (version 1)
I20240426 09:49:56.651856 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:56.658142 18307 tablet.cc:1722] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:56.658460 18307 tablet.cc:1620] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:56.658778 18471 tablet.cc:1722] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:56.659087 18471 tablet.cc:1620] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:56.659224 18308 tablet.cc:1722] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:56.659488 18308 tablet.cc:1620] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:56.664536 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=54beaf465fd943cfb6f361d5a6c661f1] alter complete (version 2)
I20240426 09:49:56.693260 17755 raft_consensus.cc:482] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:56.693406 17755 raft_consensus.cc:504] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.693810 17755 leader_election.cc:288] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:56.694155 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9726ca8b641178824a7fa5977bbdf" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:56.694279 6880 raft_consensus.cc:2454] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:56.694264 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9726ca8b641178824a7fa5977bbdf" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:56.694376 9416 raft_consensus.cc:2454] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:56.694578 9484 leader_election.cc:302] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:56.694895 17755 raft_consensus.cc:2781] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:56.694944 17755 raft_consensus.cc:482] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:56.694974 17755 raft_consensus.cc:3037] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.696022 17755 raft_consensus.cc:504] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.696328 17755 leader_election.cc:288] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:56.696772 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9726ca8b641178824a7fa5977bbdf" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:56.696870 9416 raft_consensus.cc:3037] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.697357 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9726ca8b641178824a7fa5977bbdf" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:56.697455 6880 raft_consensus.cc:3037] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.697723 9416 raft_consensus.cc:2454] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:56.697973 9482 leader_election.cc:302] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:56.698323 6880 raft_consensus.cc:2454] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:56.698498 17387 raft_consensus.cc:2781] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:56.698632 17387 raft_consensus.cc:686] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:56.698779 17387 consensus_queue.cc:237] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:56.700995 6738 catalog_manager.cc:5484] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:56.731834 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:56.734349 9661 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:56.738229 6992 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:49:56.739578 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:56.739806 7255 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:56.751623 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:56.756431 9527 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:56.762064 9396 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:56.768329 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:56.775679 18307 tablet.cc:1722] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:56.778364 18471 tablet.cc:1722] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:56.781992 18308 tablet.cc:1722] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:56.784636 6731 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=54beaf465fd943cfb6f361d5a6c661f1] alter complete (version 3)
I20240426 09:49:56.816890 9669 raft_consensus.cc:3037] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:56.819468 17905 consensus_queue.cc:1035] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.824729 18339 consensus_queue.cc:1035] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.837579 18339 consensus_queue.cc:1035] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.840549 18339 consensus_queue.cc:1035] T 3086b0bb58174f4db7c29db9a74dae88 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:56.887004 6738 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:56.887187 6738 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:56.889886 6738 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet a1e3e872ead24205ab597843fd8e167f
I20240426 09:49:56.890378 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e3e872ead24205ab597843fd8e167f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:56 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:56.890628 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e3e872ead24205ab597843fd8e167f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:56 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:56.890851 7275 raft_consensus.cc:1261] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:49:56.891222 18472 consensus_queue.cc:1035] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:56.891400 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e3e872ead24205ab597843fd8e167f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:56 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:56.891875 18537 tablet_replica.cc:330] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:56.892038 18537 raft_consensus.cc:2229] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:56.892190 18537 raft_consensus.cc:2258] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:56.892416 18538 tablet_replica.cc:330] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:56.892933 18537 ts_tablet_manager.cc:1875] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:56.895041 18538 raft_consensus.cc:2229] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:56.895202 18538 raft_consensus.cc:2258] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:56.902887 18538 ts_tablet_manager.cc:1875] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:56.904122 18539 tablet_replica.cc:330] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:56.904590 18539 raft_consensus.cc:2229] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:56.905735 18539 raft_consensus.cc:2258] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:56.906410 18539 ts_tablet_manager.cc:1875] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:56.912990 18538 ts_tablet_manager.cc:1888] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:56.913161 18538 log.cc:1196] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/a1e3e872ead24205ab597843fd8e167f
I20240426 09:49:56.914885 18538 ts_tablet_manager.cc:1909] T a1e3e872ead24205ab597843fd8e167f P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:56.915834 18539 ts_tablet_manager.cc:1888] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:56.916635 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet a1e3e872ead24205ab597843fd8e167f (table rebalancer_dynamic_table_new_name [id=54beaf465fd943cfb6f361d5a6c661f1]) successfully deleted
I20240426 09:49:56.917903 18539 log.cc:1196] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/a1e3e872ead24205ab597843fd8e167f
I20240426 09:49:56.920493 18539 ts_tablet_manager.cc:1909] T a1e3e872ead24205ab597843fd8e167f P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:56.920599 18537 ts_tablet_manager.cc:1888] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:56.920737 18537 log.cc:1196] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/a1e3e872ead24205ab597843fd8e167f
I20240426 09:49:56.921069 18537 ts_tablet_manager.cc:1909] T a1e3e872ead24205ab597843fd8e167f P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:56.921258 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet a1e3e872ead24205ab597843fd8e167f (table rebalancer_dynamic_table_new_name [id=54beaf465fd943cfb6f361d5a6c661f1]) successfully deleted
I20240426 09:49:56.921864 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet a1e3e872ead24205ab597843fd8e167f (table rebalancer_dynamic_table_new_name [id=54beaf465fd943cfb6f361d5a6c661f1]) successfully deleted
I20240426 09:49:56.922353 6880 raft_consensus.cc:1261] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:49:56.922849 18472 consensus_queue.cc:1035] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:56.926223 7011 raft_consensus.cc:1261] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:49:56.926790 18472 consensus_queue.cc:1035] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:56.980142 6992 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:49:56.986503 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:56.988786 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:56.992947 6731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:57.001101 7255 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:57.001989 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:57.004092 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 8cad402972ea4bc0b0bad0616d38aa63 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7c08cfee8ef24b2c9b4e58c0f85f271a]), partition=
I20240426 09:49:57.004382 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cad402972ea4bc0b0bad0616d38aa63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.004539 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 8cad402972ea4bc0b0bad0616d38aa63 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7c08cfee8ef24b2c9b4e58c0f85f271a]), partition=
I20240426 09:49:57.004747 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cad402972ea4bc0b0bad0616d38aa63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.004715 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 8cad402972ea4bc0b0bad0616d38aa63 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7c08cfee8ef24b2c9b4e58c0f85f271a]), partition=
I20240426 09:49:57.004940 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cad402972ea4bc0b0bad0616d38aa63. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.017608 9396 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:57.026901 9527 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:57.029974 18561 tablet_bootstrap.cc:492] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:57.030234 18562 tablet_bootstrap.cc:492] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:57.032843 9661 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:57.033002 18562 tablet_bootstrap.cc:654] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.034029 18561 tablet_bootstrap.cc:654] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.036964 18563 tablet_bootstrap.cc:492] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:57.037854 18563 tablet_bootstrap.cc:654] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.039072 18561 tablet_bootstrap.cc:492] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:57.039165 18561 ts_tablet_manager.cc:1367] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240426 09:49:57.039592 18561 raft_consensus.cc:348] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.039700 18561 raft_consensus.cc:374] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.039736 18561 raft_consensus.cc:729] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.039861 18561 consensus_queue.cc:260] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.040128 18561 ts_tablet_manager.cc:1398] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:57.040856 18563 tablet_bootstrap.cc:492] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:57.040993 18563 ts_tablet_manager.cc:1367] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:57.041648 18563 raft_consensus.cc:348] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.041743 18563 raft_consensus.cc:374] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.041777 18563 raft_consensus.cc:729] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.041926 18563 consensus_queue.cc:260] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.042200 18563 ts_tablet_manager.cc:1398] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:57.065938 18562 tablet_bootstrap.cc:492] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:57.066076 18562 ts_tablet_manager.cc:1367] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20240426 09:49:57.066571 18562 raft_consensus.cc:348] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.066699 18562 raft_consensus.cc:374] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.066732 18562 raft_consensus.cc:729] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.066931 18562 consensus_queue.cc:260] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.067226 18562 ts_tablet_manager.cc:1398] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:57.074285 17777 raft_consensus.cc:482] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:57.074388 17777 raft_consensus.cc:504] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.074756 17777 leader_election.cc:288] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:57.075167 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cad402972ea4bc0b0bad0616d38aa63" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:57.075301 6880 raft_consensus.cc:2454] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:57.075270 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cad402972ea4bc0b0bad0616d38aa63" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:57.075408 9548 raft_consensus.cc:2454] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:57.075582 9616 leader_election.cc:302] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:57.075755 17777 raft_consensus.cc:2781] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:57.075827 17777 raft_consensus.cc:482] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:57.075861 17777 raft_consensus.cc:3037] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.076951 17777 raft_consensus.cc:504] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.077263 17777 leader_election.cc:288] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:57.077529 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cad402972ea4bc0b0bad0616d38aa63" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:57.077631 9549 raft_consensus.cc:3037] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.077739 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cad402972ea4bc0b0bad0616d38aa63" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:57.077847 6880 raft_consensus.cc:3037] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.078583 9549 raft_consensus.cc:2454] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:57.078881 6880 raft_consensus.cc:2454] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:57.079166 9616 leader_election.cc:302] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:57.079586 17668 raft_consensus.cc:2781] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:57.079660 17668 raft_consensus.cc:686] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:57.079775 17668 consensus_queue.cc:237] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.082494 6731 catalog_manager.cc:5484] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:57.096697 9285 consensus_queue.cc:237] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:57.097945 7143 raft_consensus.cc:1261] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:57.098228 7275 raft_consensus.cc:1261] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:57.098486 17598 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:57.098851 17598 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:57.100598 18586 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:57.100908 7143 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:57.100976 7275 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:57.104635 6731 catalog_manager.cc:5484] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:57.107946 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 279519370ee640fd9bbd3028008f1d5d with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:57.108243 18592 tablet_replica.cc:330] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:57.108422 18592 raft_consensus.cc:2229] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:57.108603 18592 raft_consensus.cc:2258] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:57.109035 18592 ts_tablet_manager.cc:1875] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:57.111272 18592 ts_tablet_manager.cc:1888] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:57.111364 18592 log.cc:1196] T 279519370ee640fd9bbd3028008f1d5d P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/279519370ee640fd9bbd3028008f1d5d
I20240426 09:49:57.111836 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 279519370ee640fd9bbd3028008f1d5d (table rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]) successfully deleted
I20240426 09:49:57.152460 18472 consensus_queue.cc:1035] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:57.162091 17387 consensus_queue.cc:1035] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:57.196359 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:57.201884 9549 raft_consensus.cc:1261] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:57.201949 6880 raft_consensus.cc:1261] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:57.202281 17777 consensus_queue.cc:1035] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:57.203038 17777 consensus_queue.cc:1035] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:57.206421 6734 catalog_manager.cc:5576] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571) does not have the latest schema for tablet 8cad402972ea4bc0b0bad0616d38aa63 (table rebalancer_dynamic_table [id=7c08cfee8ef24b2c9b4e58c0f85f271a]). Expected version 1 got 0
I20240426 09:49:57.208796 18471 tablet.cc:1722] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:57.209069 18471 tablet.cc:1620] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:57.208654 18633 tablet.cc:1722] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:57.211443 18633 tablet.cc:1620] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:57.212085 18634 tablet.cc:1722] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:57.212396 18634 tablet.cc:1620] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:57.213474 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=7c08cfee8ef24b2c9b4e58c0f85f271a] alter complete (version 1)
I20240426 09:49:57.220163 6738 catalog_manager.cc:5576] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979) does not have the latest schema for tablet 8cad402972ea4bc0b0bad0616d38aa63 (table rebalancer_dynamic_table [id=7c08cfee8ef24b2c9b4e58c0f85f271a]). Expected version 1 got 0
I20240426 09:49:57.262471 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:57.273885 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:57.279243 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 7e494cf4a72945148ad4afa658ff1b03 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:57.279356 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 03489b207ed042bba1cab08aff3afd67 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:57.279446 9264 tablet_service.cc:1467] Processing CreateTablet for tablet 7eecd02c620e4b1eb17548a94e6a9ac1 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:57.279569 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e494cf4a72945148ad4afa658ff1b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.279640 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03489b207ed042bba1cab08aff3afd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.279686 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eecd02c620e4b1eb17548a94e6a9ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.279919 9265 tablet_service.cc:1467] Processing CreateTablet for tablet 7e494cf4a72945148ad4afa658ff1b03 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:57.280118 9265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e494cf4a72945148ad4afa658ff1b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.281224 7255 tablet_service.cc:1467] Processing CreateTablet for tablet 03489b207ed042bba1cab08aff3afd67 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:57.281469 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03489b207ed042bba1cab08aff3afd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.281555 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 7eecd02c620e4b1eb17548a94e6a9ac1 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:57.282097 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eecd02c620e4b1eb17548a94e6a9ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.283011 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 7eecd02c620e4b1eb17548a94e6a9ac1 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:57.283270 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eecd02c620e4b1eb17548a94e6a9ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.288832 9266 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:57.290060 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 7e494cf4a72945148ad4afa658ff1b03 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:57.290302 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e494cf4a72945148ad4afa658ff1b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.291656 6992 tablet_service.cc:1467] Processing CreateTablet for tablet 03489b207ed042bba1cab08aff3afd67 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:57.291929 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03489b207ed042bba1cab08aff3afd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.292232 6860 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:57.294222 18563 tablet_bootstrap.cc:492] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:57.302493 18563 tablet_bootstrap.cc:654] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.307263 18562 tablet_bootstrap.cc:492] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:57.308656 18562 tablet_bootstrap.cc:654] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.312913 18648 tablet_bootstrap.cc:492] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:57.316977 6739 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:57.319595 18562 tablet_bootstrap.cc:492] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:57.319716 18562 ts_tablet_manager.cc:1367] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20240426 09:49:57.320284 18562 raft_consensus.cc:348] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.320441 18562 raft_consensus.cc:374] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.320477 18562 raft_consensus.cc:729] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.320612 18562 consensus_queue.cc:260] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.320923 18562 ts_tablet_manager.cc:1398] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:57.325928 18633 tablet.cc:1722] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:57.326215 18633 tablet.cc:1620] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:57.326880 18471 tablet.cc:1722] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:57.327145 18471 tablet.cc:1620] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: MemRowSet was empty: no flush needed.
I20240426 09:49:57.331943 9527 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:57.336935 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=7c08cfee8ef24b2c9b4e58c0f85f271a] alter complete (version 2)
I20240426 09:49:57.339368 18634 tablet.cc:1722] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:57.339707 18634 tablet.cc:1620] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:57.341905 7254 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:57.344439 9661 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:57.347038 18646 tablet_bootstrap.cc:492] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:57.347538 18650 tablet_bootstrap.cc:492] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:57.347783 18651 tablet_bootstrap.cc:492] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:57.355295 18648 tablet_bootstrap.cc:654] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.359720 18563 tablet_bootstrap.cc:492] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:57.359841 18563 ts_tablet_manager.cc:1367] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.000s
I20240426 09:49:57.360317 18563 raft_consensus.cc:348] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:57.360399 9396 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:57.360435 18563 raft_consensus.cc:374] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.360463 18563 raft_consensus.cc:729] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.360584 18563 consensus_queue.cc:260] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:57.360860 18563 ts_tablet_manager.cc:1398] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:57.369719 18646 tablet_bootstrap.cc:654] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.372588 18646 tablet_bootstrap.cc:492] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:57.372715 18646 ts_tablet_manager.cc:1367] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.003s
I20240426 09:49:57.373385 18646 raft_consensus.cc:348] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.373538 18646 raft_consensus.cc:374] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.373580 18646 raft_consensus.cc:729] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.373760 18646 consensus_queue.cc:260] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.374148 18646 ts_tablet_manager.cc:1398] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240426 09:49:57.366950 18648 tablet_bootstrap.cc:492] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:57.382161 18648 ts_tablet_manager.cc:1367] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.069s user 0.003s sys 0.000s
I20240426 09:49:57.382725 18648 raft_consensus.cc:348] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:57.383042 18648 raft_consensus.cc:374] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.383116 18648 raft_consensus.cc:729] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.383296 18648 consensus_queue.cc:260] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:57.384536 18648 ts_tablet_manager.cc:1398] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:57.384944 18648 tablet_bootstrap.cc:492] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:57.385664 6992 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:49:57.388365 18648 tablet_bootstrap.cc:654] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.392987 17668 raft_consensus.cc:482] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:57.393074 17668 raft_consensus.cc:504] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:57.393405 17668 leader_election.cc:288] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:57.394299 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e494cf4a72945148ad4afa658ff1b03" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:57.394423 9284 raft_consensus.cc:2454] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:57.394843 9615 leader_election.cc:302] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:57.395053 17668 raft_consensus.cc:2781] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:57.395097 17668 raft_consensus.cc:482] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:57.395129 17668 raft_consensus.cc:3037] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.395288 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e494cf4a72945148ad4afa658ff1b03" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
W20240426 09:49:57.395792 9616 leader_election.cc:341] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:57.396138 17668 raft_consensus.cc:504] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:57.396446 17668 leader_election.cc:288] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:57.396941 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e494cf4a72945148ad4afa658ff1b03" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
W20240426 09:49:57.397272 9616 leader_election.cc:341] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:49:57.398977 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e494cf4a72945148ad4afa658ff1b03" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:57.399076 9286 raft_consensus.cc:3037] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.400033 9286 raft_consensus.cc:2454] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:57.400452 9615 leader_election.cc:302] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters: e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:49:57.400944 17668 raft_consensus.cc:2781] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:57.401027 17668 raft_consensus.cc:686] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:57.401156 17668 consensus_queue.cc:237] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:57.403587 6739 catalog_manager.cc:5484] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:57.418718 18651 tablet_bootstrap.cc:654] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.419209 18650 tablet_bootstrap.cc:654] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.425413 18648 tablet_bootstrap.cc:492] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:57.425515 18648 ts_tablet_manager.cc:1367] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20240426 09:49:57.425947 18648 raft_consensus.cc:348] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:57.426070 18648 raft_consensus.cc:374] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.426097 18648 raft_consensus.cc:729] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.426206 18648 consensus_queue.cc:260] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:57.426455 18648 ts_tablet_manager.cc:1398] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:57.433136 18651 tablet_bootstrap.cc:492] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:57.433307 18651 ts_tablet_manager.cc:1367] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.086s user 0.003s sys 0.000s
I20240426 09:49:57.433954 18651 raft_consensus.cc:348] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:57.434155 18651 raft_consensus.cc:374] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.434223 18651 raft_consensus.cc:729] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.434402 18651 consensus_queue.cc:260] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:57.434741 18651 ts_tablet_manager.cc:1398] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:57.440979 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:57.448930 18650 tablet_bootstrap.cc:492] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:57.449054 18650 ts_tablet_manager.cc:1367] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.102s user 0.002s sys 0.000s
I20240426 09:49:57.449575 18650 raft_consensus.cc:348] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.449735 18650 raft_consensus.cc:374] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.449775 18650 raft_consensus.cc:729] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.449901 18650 consensus_queue.cc:260] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.450273 18650 ts_tablet_manager.cc:1398] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:57.450503 18650 tablet_bootstrap.cc:492] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:57.451774 18633 tablet.cc:1722] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:57.452663 18650 tablet_bootstrap.cc:654] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.452836 18634 tablet.cc:1722] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:57.459690 18471 tablet.cc:1722] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:57.463322 18650 tablet_bootstrap.cc:492] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:57.463441 18650 ts_tablet_manager.cc:1367] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:57.463903 18650 raft_consensus.cc:348] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:57.464063 18650 raft_consensus.cc:374] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.464102 18650 raft_consensus.cc:729] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.464284 18650 consensus_queue.cc:260] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:57.468073 6739 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=7c08cfee8ef24b2c9b4e58c0f85f271a] alter complete (version 3)
I20240426 09:49:57.474900 18650 ts_tablet_manager.cc:1398] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20240426 09:49:57.475090 18650 tablet_bootstrap.cc:492] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:57.479913 18650 tablet_bootstrap.cc:654] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.481734 18650 tablet_bootstrap.cc:492] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:57.481833 18650 ts_tablet_manager.cc:1367] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20240426 09:49:57.482724 18650 raft_consensus.cc:348] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:57.482897 18650 raft_consensus.cc:374] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.482941 18650 raft_consensus.cc:729] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.483085 18650 consensus_queue.cc:260] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:57.483464 18650 ts_tablet_manager.cc:1398] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:57.494549 9672 consensus_queue.cc:237] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:57.496467 7011 raft_consensus.cc:1261] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:57.496519 6881 raft_consensus.cc:1261] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:57.497025 17668 consensus_queue.cc:1035] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:57.497972 17668 consensus_queue.cc:1035] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:57.499490 18470 raft_consensus.cc:482] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:57.499610 18470 raft_consensus.cc:504] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:57.499806 17668 raft_consensus.cc:2932] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:57.500111 18470 leader_election.cc:288] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:57.500607 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eecd02c620e4b1eb17548a94e6a9ac1" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:57.500770 9286 raft_consensus.cc:2454] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:57.501436 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eecd02c620e4b1eb17548a94e6a9ac1" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:57.501559 7274 raft_consensus.cc:2454] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:57.501893 7079 leader_election.cc:302] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:57.502051 18470 raft_consensus.cc:2781] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:57.502097 18470 raft_consensus.cc:482] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:57.502132 18470 raft_consensus.cc:3037] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.502673 7011 raft_consensus.cc:2932] T 67e3f93c0a2944e38b7542d1439c907e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:57.503283 18470 raft_consensus.cc:504] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:57.503660 18470 leader_election.cc:288] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:57.505394 6738 catalog_manager.cc:5484] T 67e3f93c0a2944e38b7542d1439c907e P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:57.506314 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eecd02c620e4b1eb17548a94e6a9ac1" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:57.506424 7274 raft_consensus.cc:3037] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.507576 7274 raft_consensus.cc:2454] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:57.508380 7079 leader_election.cc:302] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:57.509449 18470 raft_consensus.cc:2781] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:57.509536 18470 raft_consensus.cc:686] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:57.509691 18470 consensus_queue.cc:237] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:57.510460 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eecd02c620e4b1eb17548a94e6a9ac1" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:57.510587 9284 raft_consensus.cc:3037] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.511682 9284 raft_consensus.cc:2454] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:57.511632 6881 raft_consensus.cc:2932] T 67e3f93c0a2944e38b7542d1439c907e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:57.512984 6739 catalog_manager.cc:5484] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:57.515410 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 67e3f93c0a2944e38b7542d1439c907e with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:57.517432 18684 tablet_replica.cc:330] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:57.517657 18684 raft_consensus.cc:2229] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:57.517786 18684 raft_consensus.cc:2258] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:57.518206 18684 ts_tablet_manager.cc:1875] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:57.520371 18684 ts_tablet_manager.cc:1888] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:57.520491 18684 log.cc:1196] T 67e3f93c0a2944e38b7542d1439c907e P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/67e3f93c0a2944e38b7542d1439c907e
I20240426 09:49:57.521906 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 67e3f93c0a2944e38b7542d1439c907e (table rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]) successfully deleted
I20240426 09:49:57.570492 6739 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:57.570662 6739 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:57.573762 6739 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 8cad402972ea4bc0b0bad0616d38aa63
I20240426 09:49:57.574429 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cad402972ea4bc0b0bad0616d38aa63 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:57 UTC) from {username='slave'} at 127.0.0.1:44196
I20240426 09:49:57.574472 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cad402972ea4bc0b0bad0616d38aa63 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:57 UTC) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:57.574702 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cad402972ea4bc0b0bad0616d38aa63 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:57 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:57.575520 18697 tablet_replica.cc:330] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:57.576176 18697 raft_consensus.cc:2229] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:57.576391 18697 raft_consensus.cc:2258] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:57.577093 18697 ts_tablet_manager.cc:1875] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:57.581781 18696 tablet_replica.cc:330] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:49:57.582008 18696 raft_consensus.cc:2229] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:57.582213 18696 raft_consensus.cc:2258] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:57.582629 18697 ts_tablet_manager.cc:1888] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:57.582777 18697 log.cc:1196] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/8cad402972ea4bc0b0bad0616d38aa63
I20240426 09:49:57.582967 18696 ts_tablet_manager.cc:1875] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:57.583124 18697 ts_tablet_manager.cc:1909] T 8cad402972ea4bc0b0bad0616d38aa63 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting consensus metadata
I20240426 09:49:57.584196 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 8cad402972ea4bc0b0bad0616d38aa63 (table rebalancer_dynamic_table_new_name [id=7c08cfee8ef24b2c9b4e58c0f85f271a]) successfully deleted
I20240426 09:49:57.581643 18698 tablet_replica.cc:330] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:57.584900 18698 raft_consensus.cc:2229] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:57.585139 18698 raft_consensus.cc:2258] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:57.586127 18698 ts_tablet_manager.cc:1875] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:57.588337 18698 ts_tablet_manager.cc:1888] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:57.588400 18698 log.cc:1196] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/8cad402972ea4bc0b0bad0616d38aa63
I20240426 09:49:57.591084 18698 ts_tablet_manager.cc:1909] T 8cad402972ea4bc0b0bad0616d38aa63 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:57.591389 18696 ts_tablet_manager.cc:1888] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:57.591486 18696 log.cc:1196] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/8cad402972ea4bc0b0bad0616d38aa63
I20240426 09:49:57.591832 18696 ts_tablet_manager.cc:1909] T 8cad402972ea4bc0b0bad0616d38aa63 P c244e02f3cf847c0a5b69e1059f1071c: Deleting consensus metadata
I20240426 09:49:57.595448 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 8cad402972ea4bc0b0bad0616d38aa63 (table rebalancer_dynamic_table_new_name [id=7c08cfee8ef24b2c9b4e58c0f85f271a]) successfully deleted
I20240426 09:49:57.596241 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 8cad402972ea4bc0b0bad0616d38aa63 (table rebalancer_dynamic_table_new_name [id=7c08cfee8ef24b2c9b4e58c0f85f271a]) successfully deleted
I20240426 09:49:57.675984 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:57.681273 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 21ddd767c312416881665d6ca6ec3838 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=45796408ef2d48e58590ac7198e19f99]), partition=
I20240426 09:49:57.681273 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 21ddd767c312416881665d6ca6ec3838 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=45796408ef2d48e58590ac7198e19f99]), partition=
I20240426 09:49:57.681545 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ddd767c312416881665d6ca6ec3838. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.681545 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ddd767c312416881665d6ca6ec3838. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.681545 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 21ddd767c312416881665d6ca6ec3838 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=45796408ef2d48e58590ac7198e19f99]), partition=
I20240426 09:49:57.681779 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ddd767c312416881665d6ca6ec3838. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:57.683831 18651 tablet_bootstrap.cc:492] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:57.684690 18726 tablet_bootstrap.cc:492] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:57.684805 18651 tablet_bootstrap.cc:654] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.685470 18726 tablet_bootstrap.cc:654] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.687927 18651 tablet_bootstrap.cc:492] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:57.688005 18651 ts_tablet_manager.cc:1367] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:57.688369 18651 raft_consensus.cc:348] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:57.688464 18651 raft_consensus.cc:374] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.688486 18651 raft_consensus.cc:729] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.688599 18651 consensus_queue.cc:260] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:57.689134 18651 ts_tablet_manager.cc:1398] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:57.690909 18563 tablet_bootstrap.cc:492] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:57.691780 18563 tablet_bootstrap.cc:654] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:57.692982 18563 tablet_bootstrap.cc:492] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:57.693075 18563 ts_tablet_manager.cc:1367] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:57.693620 18563 raft_consensus.cc:348] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:57.693755 18563 raft_consensus.cc:374] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.693800 18563 raft_consensus.cc:729] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.693923 18563 consensus_queue.cc:260] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:57.694216 18563 ts_tablet_manager.cc:1398] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:57.694386 16849 raft_consensus.cc:482] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:57.694474 16849 raft_consensus.cc:504] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.694873 16849 leader_election.cc:288] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:57.695109 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03489b207ed042bba1cab08aff3afd67" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:57.695250 7274 raft_consensus.cc:2454] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:57.695469 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03489b207ed042bba1cab08aff3afd67" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:57.695582 6881 raft_consensus.cc:2454] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:57.696511 6948 leader_election.cc:302] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:57.697304 16849 raft_consensus.cc:2781] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:57.697365 16849 raft_consensus.cc:482] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:57.697405 16849 raft_consensus.cc:3037] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.698309 16849 raft_consensus.cc:504] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.698635 16849 leader_election.cc:288] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:49:57.699357 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03489b207ed042bba1cab08aff3afd67" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:57.699462 7274 raft_consensus.cc:3037] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.700286 7274 raft_consensus.cc:2454] T 03489b207ed042bba1cab08aff3afd67 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:57.700662 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03489b207ed042bba1cab08aff3afd67" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:57.700752 6948 leader_election.cc:302] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:57.700784 6881 raft_consensus.cc:3037] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.701033 18261 raft_consensus.cc:2781] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:57.701093 18261 raft_consensus.cc:686] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:57.701197 18261 consensus_queue.cc:237] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:57.701644 6881 raft_consensus.cc:2454] T 03489b207ed042bba1cab08aff3afd67 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:57.703543 6735 catalog_manager.cc:5484] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:57.706401 18726 tablet_bootstrap.cc:492] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:57.706538 18726 ts_tablet_manager.cc:1367] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240426 09:49:57.707033 18726 raft_consensus.cc:348] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:57.707147 18726 raft_consensus.cc:374] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:57.707207 18726 raft_consensus.cc:729] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:57.707331 18726 consensus_queue.cc:260] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:57.707692 18726 ts_tablet_manager.cc:1398] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:57.736868 9661 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:57.737601 6992 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240426 09:49:57.738214 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:57.739130 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:57.739590 9527 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:57.743127 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:57.744999 7254 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:57.778447 17668 consensus_queue.cc:1035] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:57.781690 7274 raft_consensus.cc:3037] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:57.783443 17777 consensus_queue.cc:1035] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:57.787176 9396 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:57.840940 6881 consensus_queue.cc:237] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:57.842180 7274 raft_consensus.cc:1261] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:57.842314 7011 raft_consensus.cc:1261] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:57.842710 18560 consensus_queue.cc:1035] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:57.842952 18240 consensus_queue.cc:1035] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:57.844743 18765 raft_consensus.cc:2932] T 22930f3d5b514c9884359f6c939c53e3 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:57.844930 7011 raft_consensus.cc:2932] T 22930f3d5b514c9884359f6c939c53e3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:57.845055 7274 raft_consensus.cc:2932] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:57.848655 6735 catalog_manager.cc:5484] T 22930f3d5b514c9884359f6c939c53e3 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:57.852214 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 22930f3d5b514c9884359f6c939c53e3 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:57.852471 18698 tablet_replica.cc:330] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:57.852659 18698 raft_consensus.cc:2229] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:57.852792 18698 raft_consensus.cc:2258] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:57.853237 18698 ts_tablet_manager.cc:1875] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:57.855341 18698 ts_tablet_manager.cc:1888] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:57.855422 18698 log.cc:1196] T 22930f3d5b514c9884359f6c939c53e3 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/22930f3d5b514c9884359f6c939c53e3
I20240426 09:49:57.856012 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 22930f3d5b514c9884359f6c939c53e3 (table rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]) successfully deleted
I20240426 09:49:57.938843 18470 consensus_queue.cc:1035] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:57.948331 18133 consensus_queue.cc:1035] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.072760 18782 raft_consensus.cc:482] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:58.072921 18782 raft_consensus.cc:504] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:58.073314 18782 leader_election.cc:288] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:58.073634 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ddd767c312416881665d6ca6ec3838" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:58.073704 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ddd767c312416881665d6ca6ec3838" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:58.073815 7142 raft_consensus.cc:2454] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:58.073860 9669 raft_consensus.cc:2454] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:58.074122 9350 leader_election.cc:302] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:49:58.074276 18782 raft_consensus.cc:2781] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:58.074334 18782 raft_consensus.cc:482] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:58.074362 18782 raft_consensus.cc:3037] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.075515 18782 raft_consensus.cc:504] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:58.075822 18782 leader_election.cc:288] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:58.076112 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ddd767c312416881665d6ca6ec3838" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:58.076220 7142 raft_consensus.cc:3037] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.076191 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ddd767c312416881665d6ca6ec3838" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:58.076289 9669 raft_consensus.cc:3037] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.077229 7142 raft_consensus.cc:2454] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:58.077243 9669 raft_consensus.cc:2454] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:58.077484 9350 leader_election.cc:302] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:58.077672 18782 raft_consensus.cc:2781] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:58.077761 18782 raft_consensus.cc:686] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:58.077896 18782 consensus_queue.cc:237] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:49:58.080128 6739 catalog_manager.cc:5484] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:49:58.105257 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:58.107561 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:58.108906 6992 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240426 09:49:58.110296 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:58.113006 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:58.115150 7254 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:58.122907 9661 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:58.134302 18731 consensus_queue.cc:1035] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.151314 18261 consensus_queue.cc:1035] T 03489b207ed042bba1cab08aff3afd67 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.160431 9527 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:58.221812 9549 consensus_queue.cc:237] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:58.223102 9669 raft_consensus.cc:1261] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:58.223141 7142 raft_consensus.cc:1261] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:58.223492 17755 consensus_queue.cc:1035] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:58.223698 17387 consensus_queue.cc:1035] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:58.225380 18845 raft_consensus.cc:2932] T b32cacd1e7354933928c427f9646cafb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:58.225687 7142 raft_consensus.cc:2932] T b32cacd1e7354933928c427f9646cafb P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:58.225783 9669 raft_consensus.cc:2932] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:58.229158 6739 catalog_manager.cc:5484] T b32cacd1e7354933928c427f9646cafb P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:58.232882 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet b32cacd1e7354933928c427f9646cafb with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:58.233242 18846 tablet_replica.cc:330] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:58.233443 18846 raft_consensus.cc:2229] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:58.233659 18846 raft_consensus.cc:2258] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:58.234542 18846 ts_tablet_manager.cc:1875] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:58.236579 18846 ts_tablet_manager.cc:1888] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:58.236646 18846 log.cc:1196] T b32cacd1e7354933928c427f9646cafb P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/b32cacd1e7354933928c427f9646cafb
I20240426 09:49:58.237748 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet b32cacd1e7354933928c427f9646cafb (table rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]) successfully deleted
I20240426 09:49:58.260026 6736 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:58.264725 9669 raft_consensus.cc:1261] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:58.264758 7142 raft_consensus.cc:1261] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:58.265182 17511 consensus_queue.cc:1035] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.265486 17511 consensus_queue.cc:1035] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.270515 6739 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:58.270730 6736 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet 21ddd767c312416881665d6ca6ec3838 (table rebalancer_dynamic_table [id=45796408ef2d48e58590ac7198e19f99]). Expected version 1 got 0
I20240426 09:49:58.271114 6738 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet 21ddd767c312416881665d6ca6ec3838 (table rebalancer_dynamic_table [id=45796408ef2d48e58590ac7198e19f99]). Expected version 1 got 0
I20240426 09:49:58.271375 18871 tablet.cc:1722] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:58.271744 18871 tablet.cc:1620] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:58.272081 18866 tablet.cc:1722] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:58.275156 18866 tablet.cc:1620] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:58.277909 6736 catalog_manager.cc:5576] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059) does not have the latest schema for tablet 21ddd767c312416881665d6ca6ec3838 (table rebalancer_dynamic_table [id=45796408ef2d48e58590ac7198e19f99]). Expected version 1 got 0
I20240426 09:49:58.278908 18873 tablet.cc:1722] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:58.279203 18873 tablet.cc:1620] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:58.280145 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=45796408ef2d48e58590ac7198e19f99] alter complete (version 1)
I20240426 09:49:58.281803 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 7799bbe0befb41d0a155ff51f41c23ad (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:58.282045 6991 tablet_service.cc:1467] Processing CreateTablet for tablet 61bc946b265243198d1cb16247c43b7f (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:58.282121 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7799bbe0befb41d0a155ff51f41c23ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.282213 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 7799bbe0befb41d0a155ff51f41c23ad (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:58.282302 6991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61bc946b265243198d1cb16247c43b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.282467 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7799bbe0befb41d0a155ff51f41c23ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.283757 9529 tablet_service.cc:1467] Processing CreateTablet for tablet 4e344f55669d4b16a8d3d5ef274b87fe (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:58.283784 6992 tablet_service.cc:1467] Processing CreateTablet for tablet 7799bbe0befb41d0a155ff51f41c23ad (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:58.283977 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e344f55669d4b16a8d3d5ef274b87fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.284034 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7799bbe0befb41d0a155ff51f41c23ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.286317 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 61bc946b265243198d1cb16247c43b7f (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:58.286613 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61bc946b265243198d1cb16247c43b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.289151 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 61bc946b265243198d1cb16247c43b7f (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:58.289173 18877 tablet_bootstrap.cc:492] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:58.289376 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61bc946b265243198d1cb16247c43b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.289921 18876 tablet_bootstrap.cc:492] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:58.290233 18877 tablet_bootstrap.cc:654] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.290867 18876 tablet_bootstrap.cc:654] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.291375 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 4e344f55669d4b16a8d3d5ef274b87fe (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:58.291630 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e344f55669d4b16a8d3d5ef274b87fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.291815 18875 tablet_bootstrap.cc:492] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:58.292315 18878 tablet_bootstrap.cc:492] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:58.293360 18878 tablet_bootstrap.cc:654] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.293367 18875 tablet_bootstrap.cc:654] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.293429 18877 tablet_bootstrap.cc:492] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:58.293509 18877 ts_tablet_manager.cc:1367] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:49:58.294044 18877 raft_consensus.cc:348] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.294164 18877 raft_consensus.cc:374] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.294196 18877 raft_consensus.cc:729] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.294327 18877 consensus_queue.cc:260] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.294364 9264 tablet_service.cc:1467] Processing CreateTablet for tablet 4e344f55669d4b16a8d3d5ef274b87fe (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:58.294461 18876 tablet_bootstrap.cc:492] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:58.294526 18878 tablet_bootstrap.cc:492] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:58.294536 18876 ts_tablet_manager.cc:1367] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240426 09:49:58.294613 18878 ts_tablet_manager.cc:1367] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:58.294646 18877 ts_tablet_manager.cc:1398] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:58.294656 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e344f55669d4b16a8d3d5ef274b87fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.295039 18876 raft_consensus.cc:348] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.295133 18876 raft_consensus.cc:374] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.295163 18876 raft_consensus.cc:729] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.295091 18878 raft_consensus.cc:348] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.295200 18878 raft_consensus.cc:374] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.295228 18878 raft_consensus.cc:729] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.295306 18876 consensus_queue.cc:260] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.295331 18878 consensus_queue.cc:260] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.295579 18876 ts_tablet_manager.cc:1398] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:58.295581 18878 ts_tablet_manager.cc:1398] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:58.297657 18876 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:58.298619 18876 tablet_bootstrap.cc:654] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.301378 18875 tablet_bootstrap.cc:492] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:58.301457 18875 ts_tablet_manager.cc:1367] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240426 09:49:58.301888 18875 raft_consensus.cc:348] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.301993 18875 raft_consensus.cc:374] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.302023 18875 raft_consensus.cc:729] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.302127 18875 consensus_queue.cc:260] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.302388 18875 ts_tablet_manager.cc:1398] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:58.302605 18875 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:58.303856 18875 tablet_bootstrap.cc:654] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.305254 18876 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:58.305348 18876 ts_tablet_manager.cc:1367] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:49:58.305827 18876 raft_consensus.cc:348] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.305933 18876 raft_consensus.cc:374] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.305963 18876 raft_consensus.cc:729] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.306084 18876 consensus_queue.cc:260] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.306339 18876 ts_tablet_manager.cc:1398] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:58.306460 18878 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:58.307730 18878 tablet_bootstrap.cc:654] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.308504 18875 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:58.308588 18875 ts_tablet_manager.cc:1367] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:58.309119 18875 raft_consensus.cc:348] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.309211 18875 raft_consensus.cc:374] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.309244 18875 raft_consensus.cc:729] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.309374 18875 consensus_queue.cc:260] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.309629 18875 ts_tablet_manager.cc:1398] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:58.309753 18875 tablet_bootstrap.cc:492] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:58.300789 18883 tablet_bootstrap.cc:492] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:58.310590 18875 tablet_bootstrap.cc:654] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.312407 18883 tablet_bootstrap.cc:654] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.313688 18878 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:58.313802 18878 ts_tablet_manager.cc:1367] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:49:58.314059 18883 tablet_bootstrap.cc:492] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:58.314143 18883 ts_tablet_manager.cc:1367] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:58.314267 18878 raft_consensus.cc:348] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.314352 18878 raft_consensus.cc:374] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.314373 18878 raft_consensus.cc:729] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.314473 18878 consensus_queue.cc:260] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.314643 18883 raft_consensus.cc:348] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.314730 18878 ts_tablet_manager.cc:1398] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:58.314749 18883 raft_consensus.cc:374] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.314803 18883 raft_consensus.cc:729] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.315297 18875 tablet_bootstrap.cc:492] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:58.315380 18875 ts_tablet_manager.cc:1367] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:49:58.315789 18875 raft_consensus.cc:348] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.315893 18875 raft_consensus.cc:374] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.315925 18875 raft_consensus.cc:729] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.316031 18875 consensus_queue.cc:260] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.316907 18875 ts_tablet_manager.cc:1398] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:58.317301 18883 consensus_queue.cc:260] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.318848 18883 ts_tablet_manager.cc:1398] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240426 09:49:58.324132 18212 raft_consensus.cc:482] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:58.324241 18212 raft_consensus.cc:504] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.324702 18212 leader_election.cc:288] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:58.325347 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61bc946b265243198d1cb16247c43b7f" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:58.325479 7011 raft_consensus.cc:2454] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:58.325763 7208 leader_election.cc:302] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:58.325817 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61bc946b265243198d1cb16247c43b7f" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:58.325940 6880 raft_consensus.cc:2454] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:58.325938 18212 raft_consensus.cc:2781] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:58.325999 18212 raft_consensus.cc:482] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:58.326038 18212 raft_consensus.cc:3037] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.327334 18212 raft_consensus.cc:504] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.327723 18212 leader_election.cc:288] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:58.328125 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61bc946b265243198d1cb16247c43b7f" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:58.328235 6880 raft_consensus.cc:3037] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.329191 6880 raft_consensus.cc:2454] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:58.329933 7210 leader_election.cc:302] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c244e02f3cf847c0a5b69e1059f1071c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:58.330154 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61bc946b265243198d1cb16247c43b7f" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:58.330255 7011 raft_consensus.cc:3037] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.331171 7011 raft_consensus.cc:2454] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:58.331362 18212 raft_consensus.cc:2781] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:58.331467 18212 raft_consensus.cc:686] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:58.331593 18212 consensus_queue.cc:237] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.334493 6732 catalog_manager.cc:5484] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:58.388444 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:58.394520 18866 tablet.cc:1722] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:58.394892 18866 tablet.cc:1620] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:58.398442 18871 tablet.cc:1722] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:58.398785 18871 tablet.cc:1620] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:58.398945 18873 tablet.cc:1722] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:58.399214 18873 tablet.cc:1620] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: MemRowSet was empty: no flush needed.
I20240426 09:49:58.406373 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=45796408ef2d48e58590ac7198e19f99] alter complete (version 2)
I20240426 09:49:58.435142 16849 raft_consensus.cc:482] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:58.435273 16849 raft_consensus.cc:504] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.435626 16849 leader_election.cc:288] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:58.435930 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e344f55669d4b16a8d3d5ef274b87fe" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:58.435977 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e344f55669d4b16a8d3d5ef274b87fe" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:58.436076 9286 raft_consensus.cc:2454] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:58.436089 9549 raft_consensus.cc:2454] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:58.436318 6947 leader_election.cc:302] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:58.436463 16849 raft_consensus.cc:2781] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:58.436515 16849 raft_consensus.cc:482] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:58.436540 16849 raft_consensus.cc:3037] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.437346 16849 raft_consensus.cc:504] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.437609 16849 leader_election.cc:288] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:49:58.437841 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e344f55669d4b16a8d3d5ef274b87fe" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:58.437883 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e344f55669d4b16a8d3d5ef274b87fe" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:58.437930 9549 raft_consensus.cc:3037] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.437976 9286 raft_consensus.cc:3037] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.438762 9549 raft_consensus.cc:2454] T 4e344f55669d4b16a8d3d5ef274b87fe P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:58.438802 9286 raft_consensus.cc:2454] T 4e344f55669d4b16a8d3d5ef274b87fe P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:58.439066 6946 leader_election.cc:302] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:58.439204 16849 raft_consensus.cc:2781] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:58.439288 16849 raft_consensus.cc:686] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:58.439451 16849 consensus_queue.cc:237] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.441771 6740 catalog_manager.cc:5484] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:58.508051 6740 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:58.513319 18866 tablet.cc:1722] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:58.515980 18873 tablet.cc:1722] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:58.521684 18871 tablet.cc:1722] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:58.521719 6737 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=45796408ef2d48e58590ac7198e19f99] alter complete (version 3)
I20240426 09:49:58.532388 16849 raft_consensus.cc:482] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:58.532521 16849 raft_consensus.cc:504] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.533520 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7799bbe0befb41d0a155ff51f41c23ad" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:58.533644 9549 raft_consensus.cc:2454] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:58.534281 6946 leader_election.cc:302] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:58.534454 18261 raft_consensus.cc:2781] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:58.534507 18261 raft_consensus.cc:482] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:58.534547 18261 raft_consensus.cc:3037] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.534885 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7799bbe0befb41d0a155ff51f41c23ad" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:58.535015 7275 raft_consensus.cc:2454] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 0.
I20240426 09:49:58.535341 16849 leader_election.cc:288] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:58.535687 18261 raft_consensus.cc:504] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.535998 18261 leader_election.cc:288] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Requested vote from peers b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:58.536257 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7799bbe0befb41d0a155ff51f41c23ad" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:58.536361 7275 raft_consensus.cc:3037] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.536453 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7799bbe0befb41d0a155ff51f41c23ad" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:58.536556 9549 raft_consensus.cc:3037] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.537539 9549 raft_consensus.cc:2454] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:58.537564 7275 raft_consensus.cc:2454] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 1.
I20240426 09:49:58.537797 6948 leader_election.cc:302] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:58.537945 18261 raft_consensus.cc:2781] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:58.538012 18261 raft_consensus.cc:686] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:49:58.538156 18261 consensus_queue.cc:237] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.541496 6740 catalog_manager.cc:5484] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:58.571656 9661 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:58.574914 6993 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:49:58.577697 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:49:58.579088 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:58.579929 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:58.606592 9527 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:58.612146 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:58.624011 6740 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:58.624181 6740 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:58.626744 6740 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet 21ddd767c312416881665d6ca6ec3838
I20240426 09:49:58.627275 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ddd767c312416881665d6ca6ec3838 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:58 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:58.627589 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ddd767c312416881665d6ca6ec3838 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:58 UTC) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:58.627663 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 21ddd767c312416881665d6ca6ec3838 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:58 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:58.627823 18846 tablet_replica.cc:330] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:58.627980 18846 raft_consensus.cc:2229] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:58.628180 18846 raft_consensus.cc:2258] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:58.629179 18846 ts_tablet_manager.cc:1875] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:58.631170 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:58.632282 18846 ts_tablet_manager.cc:1888] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:58.632350 18846 log.cc:1196] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/21ddd767c312416881665d6ca6ec3838
I20240426 09:49:58.632591 18846 ts_tablet_manager.cc:1909] T 21ddd767c312416881665d6ca6ec3838 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:58.634307 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 21ddd767c312416881665d6ca6ec3838 (table rebalancer_dynamic_table_new_name [id=45796408ef2d48e58590ac7198e19f99]) successfully deleted
I20240426 09:49:58.658344 18924 tablet_replica.cc:330] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:58.658826 18924 raft_consensus.cc:2229] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:58.659879 18924 raft_consensus.cc:2258] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:58.660609 18924 ts_tablet_manager.cc:1875] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:58.661991 18923 tablet_replica.cc:330] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:58.662137 18923 raft_consensus.cc:2229] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:58.662261 18923 raft_consensus.cc:2258] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:58.663723 18923 ts_tablet_manager.cc:1875] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:58.664417 18924 ts_tablet_manager.cc:1888] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:58.664538 18924 log.cc:1196] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/21ddd767c312416881665d6ca6ec3838
I20240426 09:49:58.664839 18924 ts_tablet_manager.cc:1909] T 21ddd767c312416881665d6ca6ec3838 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting consensus metadata
I20240426 09:49:58.665511 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 21ddd767c312416881665d6ca6ec3838 (table rebalancer_dynamic_table_new_name [id=45796408ef2d48e58590ac7198e19f99]) successfully deleted
I20240426 09:49:58.665971 18923 ts_tablet_manager.cc:1888] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:58.666033 18923 log.cc:1196] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/21ddd767c312416881665d6ca6ec3838
I20240426 09:49:58.666294 18923 ts_tablet_manager.cc:1909] T 21ddd767c312416881665d6ca6ec3838 P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:58.669515 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 21ddd767c312416881665d6ca6ec3838 (table rebalancer_dynamic_table_new_name [id=45796408ef2d48e58590ac7198e19f99]) successfully deleted
I20240426 09:49:58.691673 7274 consensus_queue.cc:237] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:49:58.693161 6880 raft_consensus.cc:1261] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:58.693769 7011 raft_consensus.cc:1261] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:58.693873 18705 consensus_queue.cc:1035] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:58.694582 18705 consensus_queue.cc:1035] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:58.695977 18212 raft_consensus.cc:2932] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:58.696386 6880 raft_consensus.cc:2932] T 1a99c72632ab49a5b7b2c4a5b305b119 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:58.699697 6740 catalog_manager.cc:5484] T 1a99c72632ab49a5b7b2c4a5b305b119 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:58.700593 7011 raft_consensus.cc:2932] T 1a99c72632ab49a5b7b2c4a5b305b119 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:49:58.704087 9661 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a99c72632ab49a5b7b2c4a5b305b119 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:49:58.704334 18924 tablet_replica.cc:330] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:49:58.704490 18924 raft_consensus.cc:2229] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:58.704598 18924 raft_consensus.cc:2258] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:58.704985 18924 ts_tablet_manager.cc:1875] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:58.707111 18924 ts_tablet_manager.cc:1888] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:58.707186 18924 log.cc:1196] T 1a99c72632ab49a5b7b2c4a5b305b119 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/1a99c72632ab49a5b7b2c4a5b305b119
I20240426 09:49:58.707681 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 1a99c72632ab49a5b7b2c4a5b305b119 (table rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]) successfully deleted
I20240426 09:49:58.728312 6740 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:58.733397 18705 consensus_queue.cc:1035] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.735291 7123 tablet_service.cc:1467] Processing CreateTablet for tablet bcdbd52812964134b82d9b2e4c31f60e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=acb9f31f772647138524cd45b6f8cf27]), partition=
I20240426 09:49:58.735575 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcdbd52812964134b82d9b2e4c31f60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.735947 9264 tablet_service.cc:1467] Processing CreateTablet for tablet bcdbd52812964134b82d9b2e4c31f60e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=acb9f31f772647138524cd45b6f8cf27]), partition=
I20240426 09:49:58.736192 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcdbd52812964134b82d9b2e4c31f60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.738056 9396 tablet_service.cc:1467] Processing CreateTablet for tablet bcdbd52812964134b82d9b2e4c31f60e (DEFAULT_TABLE table=rebalancer_dynamic_table [id=acb9f31f772647138524cd45b6f8cf27]), partition=
I20240426 09:49:58.738250 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcdbd52812964134b82d9b2e4c31f60e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:58.738497 18883 tablet_bootstrap.cc:492] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:58.739130 18960 tablet_bootstrap.cc:492] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:58.739559 18883 tablet_bootstrap.cc:654] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.741122 18962 tablet_bootstrap.cc:492] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:58.741540 18883 tablet_bootstrap.cc:492] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:58.741623 18883 ts_tablet_manager.cc:1367] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:49:58.742075 18883 raft_consensus.cc:348] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.742187 18883 raft_consensus.cc:374] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.742218 18883 raft_consensus.cc:729] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.742347 18883 consensus_queue.cc:260] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.742653 18883 ts_tablet_manager.cc:1398] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:58.743520 18960 tablet_bootstrap.cc:654] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.744167 18212 consensus_queue.cc:1035] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.746652 18962 tablet_bootstrap.cc:654] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:58.751107 18960 tablet_bootstrap.cc:492] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:58.751281 18960 ts_tablet_manager.cc:1367] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:49:58.751758 18960 raft_consensus.cc:348] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.751885 18960 raft_consensus.cc:374] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.751914 18960 raft_consensus.cc:729] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.752034 18960 consensus_queue.cc:260] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.752327 18960 ts_tablet_manager.cc:1398] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:58.753120 18962 tablet_bootstrap.cc:492] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:58.753216 18962 ts_tablet_manager.cc:1367] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240426 09:49:58.753623 18962 raft_consensus.cc:348] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.753772 18962 raft_consensus.cc:374] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:58.753808 18962 raft_consensus.cc:729] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:58.753952 18962 consensus_queue.cc:260] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.754246 18962 ts_tablet_manager.cc:1398] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:58.854315 17598 raft_consensus.cc:482] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:58.854444 17598 raft_consensus.cc:504] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.854911 17598 leader_election.cc:288] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:58.855198 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcdbd52812964134b82d9b2e4c31f60e" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:58.855337 7142 raft_consensus.cc:2454] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:58.855381 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcdbd52812964134b82d9b2e4c31f60e" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c" is_pre_election: true
I20240426 09:49:58.855522 9417 raft_consensus.cc:2454] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 0.
I20240426 09:49:58.855595 9219 leader_election.cc:302] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:58.855775 17598 raft_consensus.cc:2781] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:58.855827 17598 raft_consensus.cc:482] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:58.855873 17598 raft_consensus.cc:3037] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.856771 17598 raft_consensus.cc:504] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.857053 17598 leader_election.cc:288] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:49:58.857335 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcdbd52812964134b82d9b2e4c31f60e" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:58.857383 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcdbd52812964134b82d9b2e4c31f60e" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:49:58.857484 7142 raft_consensus.cc:3037] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.857515 9417 raft_consensus.cc:3037] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:58.858278 7142 raft_consensus.cc:2454] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:58.858337 9417 raft_consensus.cc:2454] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 1.
I20240426 09:49:58.858537 9219 leader_election.cc:302] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:49:58.858718 17598 raft_consensus.cc:2781] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:58.858892 17598 raft_consensus.cc:686] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:49:58.859031 17598 consensus_queue.cc:237] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:58.861608 6737 catalog_manager.cc:5484] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 0 to 1, leader changed from <none> to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:58.885711 6993 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:49:58.888953 9527 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:58.895107 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:58.910725 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:58.919072 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:58.919477 9396 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:58.919667 9661 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:58.919728 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:58.969307 16849 consensus_queue.cc:1035] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.985195 18261 consensus_queue.cc:1035] T 4e344f55669d4b16a8d3d5ef274b87fe P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.991708 6737 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:58.996538 9417 raft_consensus.cc:1261] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:58.997493 7142 raft_consensus.cc:1261] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:58.997680 17598 consensus_queue.cc:1035] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:58.998163 17598 consensus_queue.cc:1035] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.001511 6737 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet bcdbd52812964134b82d9b2e4c31f60e (table rebalancer_dynamic_table [id=acb9f31f772647138524cd45b6f8cf27]). Expected version 1 got 0
I20240426 09:49:59.004405 6737 catalog_manager.cc:5576] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203) does not have the latest schema for tablet bcdbd52812964134b82d9b2e4c31f60e (table rebalancer_dynamic_table [id=acb9f31f772647138524cd45b6f8cf27]). Expected version 1 got 0
I20240426 09:49:59.005416 18866 tablet.cc:1722] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:59.005738 18866 tablet.cc:1620] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:59.006784 18871 tablet.cc:1722] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:59.007071 18871 tablet.cc:1620] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:59.009502 19016 tablet.cc:1722] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:59.009783 19016 tablet.cc:1620] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:59.011823 6733 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=acb9f31f772647138524cd45b6f8cf27] alter complete (version 1)
I20240426 09:49:59.034622 6880 consensus_queue.cc:237] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:59.035807 9285 raft_consensus.cc:1261] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:59.035974 9417 raft_consensus.cc:1261] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:59.036249 18765 consensus_queue.cc:1035] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:59.036422 18994 consensus_queue.cc:1035] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:59.037355 19011 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.038378 18765 raft_consensus.cc:2932] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:59.039657 9417 raft_consensus.cc:2932] T e110e59d534c4bbd9fbceae078bb8286 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:59.040458 18261 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.040917 9285 raft_consensus.cc:2932] T e110e59d534c4bbd9fbceae078bb8286 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:59.041841 6737 catalog_manager.cc:5484] T e110e59d534c4bbd9fbceae078bb8286 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:49:59.046154 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet e110e59d534c4bbd9fbceae078bb8286 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44728
I20240426 09:49:59.046543 19037 tablet_replica.cc:330] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:49:59.047708 19037 raft_consensus.cc:2229] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:59.048184 19037 raft_consensus.cc:2258] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:59.048707 19037 ts_tablet_manager.cc:1875] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:59.051149 19037 ts_tablet_manager.cc:1888] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:59.051280 19037 log.cc:1196] T e110e59d534c4bbd9fbceae078bb8286 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/e110e59d534c4bbd9fbceae078bb8286
I20240426 09:49:59.052021 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet e110e59d534c4bbd9fbceae078bb8286 (table rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]) successfully deleted
I20240426 09:49:59.087110 6734 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:59.096347 9661 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd97a192183433aa48531ab7885e73e (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.096431 9659 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6f0978441e4ce69c6e40805e495fd0 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.096462 7122 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6f0978441e4ce69c6e40805e495fd0 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.096552 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd97a192183433aa48531ab7885e73e (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.096722 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd97a192183433aa48531ab7885e73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.096743 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6f0978441e4ce69c6e40805e495fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.096814 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd97a192183433aa48531ab7885e73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.096889 9659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6f0978441e4ce69c6e40805e495fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.098966 18960 tablet_bootstrap.cc:492] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:59.099092 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 845d1058274a41d49b6ac67571cb5439 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.099377 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 845d1058274a41d49b6ac67571cb5439. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.099982 18960 tablet_bootstrap.cc:654] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.100811 19080 tablet_bootstrap.cc:492] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:59.101095 18960 tablet_bootstrap.cc:492] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:59.101190 18960 ts_tablet_manager.cc:1367] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:59.101631 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 6f6f0978441e4ce69c6e40805e495fd0 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.101686 18960 raft_consensus.cc:348] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.101807 18960 raft_consensus.cc:374] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.101852 19080 tablet_bootstrap.cc:654] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.101847 18960 raft_consensus.cc:729] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.101897 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6f0978441e4ce69c6e40805e495fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.101977 18960 consensus_queue.cc:260] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.102304 18960 ts_tablet_manager.cc:1398] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:59.102440 19081 tablet_bootstrap.cc:492] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:59.102901 19080 tablet_bootstrap.cc:492] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:59.103001 19080 ts_tablet_manager.cc:1367] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:59.103482 19080 raft_consensus.cc:348] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.103579 19080 raft_consensus.cc:374] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.103611 19080 raft_consensus.cc:729] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.103721 19080 consensus_queue.cc:260] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.104020 19080 ts_tablet_manager.cc:1398] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:59.104192 19080 tablet_bootstrap.cc:492] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:59.104668 19083 tablet_bootstrap.cc:492] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:59.104710 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 845d1058274a41d49b6ac67571cb5439 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.104956 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 845d1058274a41d49b6ac67571cb5439. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.105067 19080 tablet_bootstrap.cc:654] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.104984 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd97a192183433aa48531ab7885e73e (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.105319 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd97a192183433aa48531ab7885e73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.105654 19083 tablet_bootstrap.cc:654] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.106709 19083 tablet_bootstrap.cc:492] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:59.106838 19083 ts_tablet_manager.cc:1367] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:59.107039 9660 tablet_service.cc:1467] Processing CreateTablet for tablet 845d1058274a41d49b6ac67571cb5439 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.107270 9660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 845d1058274a41d49b6ac67571cb5439. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.107267 19083 raft_consensus.cc:348] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.107368 19083 raft_consensus.cc:374] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.107398 19083 raft_consensus.cc:729] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.107496 19083 consensus_queue.cc:260] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.107787 19083 ts_tablet_manager.cc:1398] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:59.113895 19081 tablet_bootstrap.cc:654] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.115351 19086 tablet_bootstrap.cc:492] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:59.115592 19080 tablet_bootstrap.cc:492] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:59.115692 19080 ts_tablet_manager.cc:1367] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20240426 09:49:59.116159 19080 raft_consensus.cc:348] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.116258 19080 raft_consensus.cc:374] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.116308 19080 raft_consensus.cc:729] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.118693 19086 tablet_bootstrap.cc:654] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.121003 18960 tablet_bootstrap.cc:492] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:59.121348 19081 tablet_bootstrap.cc:492] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:59.121454 19081 ts_tablet_manager.cc:1367] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240426 09:49:59.122215 18960 tablet_bootstrap.cc:654] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.122251 19081 raft_consensus.cc:348] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:59.122376 19081 raft_consensus.cc:374] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.122409 19081 raft_consensus.cc:729] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.122545 19081 consensus_queue.cc:260] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:59.124011 19081 ts_tablet_manager.cc:1398] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:49:59.127583 19083 tablet_bootstrap.cc:492] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:59.125582 6735 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:59.116428 19080 consensus_queue.cc:260] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.131126 19080 ts_tablet_manager.cc:1398] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.015s user 0.000s sys 0.001s
I20240426 09:49:59.131498 19080 tablet_bootstrap.cc:492] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:49:59.132485 19080 tablet_bootstrap.cc:654] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.135217 19086 tablet_bootstrap.cc:492] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:59.135318 19086 ts_tablet_manager.cc:1367] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240426 09:49:59.135802 19086 raft_consensus.cc:348] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.135905 19086 raft_consensus.cc:374] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.135941 19086 raft_consensus.cc:729] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.136052 19086 consensus_queue.cc:260] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.139391 19086 ts_tablet_manager.cc:1398] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:49:59.139991 19080 tablet_bootstrap.cc:492] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19: No bootstrap required, opened a new log
I20240426 09:49:59.140087 19080 ts_tablet_manager.cc:1367] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240426 09:49:59.140563 19080 raft_consensus.cc:348] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:59.140683 19080 raft_consensus.cc:374] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.140717 19080 raft_consensus.cc:729] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.140843 19080 consensus_queue.cc:260] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:59.141139 19080 ts_tablet_manager.cc:1398] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:59.143718 18871 tablet.cc:1722] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:59.143792 19016 tablet.cc:1722] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:59.144081 18871 tablet.cc:1620] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:59.144102 19016 tablet.cc:1620] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:59.151041 18866 tablet.cc:1722] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:59.151420 18866 tablet.cc:1620] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: MemRowSet was empty: no flush needed.
I20240426 09:49:59.153851 6735 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=acb9f31f772647138524cd45b6f8cf27] alter complete (version 2)
I20240426 09:49:59.155836 19083 tablet_bootstrap.cc:654] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.158690 18960 tablet_bootstrap.cc:492] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:59.158749 19083 tablet_bootstrap.cc:492] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:59.158984 18960 ts_tablet_manager.cc:1367] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.002s
I20240426 09:49:59.159104 19083 ts_tablet_manager.cc:1367] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240426 09:49:59.159600 18960 raft_consensus.cc:348] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:59.159734 18960 raft_consensus.cc:374] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.159768 18960 raft_consensus.cc:729] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.159894 18960 consensus_queue.cc:260] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:59.163446 18960 ts_tablet_manager.cc:1398] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240426 09:49:59.159852 19083 raft_consensus.cc:348] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.166096 19083 raft_consensus.cc:374] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.166152 19083 raft_consensus.cc:729] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.166368 19083 consensus_queue.cc:260] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.166721 19083 ts_tablet_manager.cc:1398] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.008s user 0.000s sys 0.000s
I20240426 09:49:59.179975 18212 raft_consensus.cc:482] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:59.180104 18212 raft_consensus.cc:504] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.180519 18212 leader_election.cc:288] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:59.181577 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd97a192183433aa48531ab7885e73e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" is_pre_election: true
I20240426 09:49:59.181721 9549 raft_consensus.cc:2454] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:59.181968 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd97a192183433aa48531ab7885e73e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:59.182148 9669 raft_consensus.cc:2454] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 0.
I20240426 09:49:59.183157 7208 leader_election.cc:302] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:59.184056 18212 raft_consensus.cc:2781] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:59.184121 18212 raft_consensus.cc:482] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:59.184160 18212 raft_consensus.cc:3037] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.185655 18212 raft_consensus.cc:504] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.186084 18212 leader_election.cc:288] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:49:59.187654 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd97a192183433aa48531ab7885e73e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:59.187762 9669 raft_consensus.cc:3037] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.188820 9669 raft_consensus.cc:2454] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:59.190341 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd97a192183433aa48531ab7885e73e" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:49:59.190476 9549 raft_consensus.cc:3037] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.191534 9549 raft_consensus.cc:2454] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 1.
I20240426 09:49:59.192323 7208 leader_election.cc:302] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:59.192641 18705 raft_consensus.cc:2781] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:59.192708 18705 raft_consensus.cc:686] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:49:59.192837 18705 consensus_queue.cc:237] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.198630 6734 catalog_manager.cc:5484] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:59.260962 6738 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240426 09:49:59.267155 19016 tablet.cc:1722] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:59.267787 18871 tablet.cc:1722] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:59.267890 18866 tablet.cc:1722] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240426 09:49:59.273572 6734 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table_new_name [id=acb9f31f772647138524cd45b6f8cf27] alter complete (version 3)
I20240426 09:49:59.286290 18652 raft_consensus.cc:482] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:59.286397 18652 raft_consensus.cc:504] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.286892 18652 leader_election.cc:288] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:59.287149 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6f0978441e4ce69c6e40805e495fd0" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:59.287153 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6f0978441e4ce69c6e40805e495fd0" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:59.287276 7142 raft_consensus.cc:2454] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:59.287276 7274 raft_consensus.cc:2454] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 0.
I20240426 09:49:59.287505 9616 leader_election.cc:302] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:59.287645 18652 raft_consensus.cc:2781] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:59.287710 18652 raft_consensus.cc:482] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:59.287751 18652 raft_consensus.cc:3037] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.288631 18652 raft_consensus.cc:504] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.288887 18652 leader_election.cc:288] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:59.289134 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6f0978441e4ce69c6e40805e495fd0" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:59.289149 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f6f0978441e4ce69c6e40805e495fd0" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:59.289250 7142 raft_consensus.cc:3037] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.289314 7274 raft_consensus.cc:3037] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.290026 7142 raft_consensus.cc:2454] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:59.290063 7274 raft_consensus.cc:2454] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 1.
I20240426 09:49:59.290385 9616 leader_election.cc:302] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:59.290546 18652 raft_consensus.cc:2781] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:59.291342 18652 raft_consensus.cc:686] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:49:59.291507 18652 consensus_queue.cc:237] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.293498 6733 catalog_manager.cc:5484] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:49:59.296416 18470 raft_consensus.cc:482] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:59.296510 18470 raft_consensus.cc:504] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:59.296900 18470 leader_election.cc:288] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:59.297222 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "845d1058274a41d49b6ac67571cb5439" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:59.297222 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "845d1058274a41d49b6ac67571cb5439" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19" is_pre_election: true
I20240426 09:49:59.297346 9669 raft_consensus.cc:2454] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:59.297371 6880 raft_consensus.cc:2454] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 0.
I20240426 09:49:59.297631 7079 leader_election.cc:302] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:59.297786 18470 raft_consensus.cc:2781] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:59.297843 18470 raft_consensus.cc:482] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:59.297873 18470 raft_consensus.cc:3037] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.298862 18470 raft_consensus.cc:504] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:59.299144 18470 leader_election.cc:288] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:49:59.299510 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "845d1058274a41d49b6ac67571cb5439" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:49:59.299512 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "845d1058274a41d49b6ac67571cb5439" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:59.299602 9669 raft_consensus.cc:3037] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.299618 6880 raft_consensus.cc:3037] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.300518 9669 raft_consensus.cc:2454] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:59.300611 6880 raft_consensus.cc:2454] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 1.
I20240426 09:49:59.300830 7077 leader_election.cc:302] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19; no voters:
I20240426 09:49:59.300987 18470 raft_consensus.cc:2781] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:59.301092 18470 raft_consensus.cc:686] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:49:59.301220 18470 consensus_queue.cc:237] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:49:59.303360 6733 catalog_manager.cc:5484] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:49:59.329034 6993 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:49:59.329627 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:49:59.333650 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:59.338034 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:59.342264 9527 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:59.343961 9396 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:59.346941 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:49:59.375808 6733 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:59.375980 6733 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240426 09:49:59.378244 6733 catalog_manager.cc:5771] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: Sending DeleteTablet for 3 replicas of tablet bcdbd52812964134b82d9b2e4c31f60e
I20240426 09:49:59.378971 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet bcdbd52812964134b82d9b2e4c31f60e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:59 UTC) from {username='slave'} at 127.0.0.1:52544
I20240426 09:49:59.379094 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet bcdbd52812964134b82d9b2e4c31f60e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:59 UTC) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:59.379325 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet bcdbd52812964134b82d9b2e4c31f60e with delete_type TABLET_DATA_DELETED (Table deleted at 2024-04-26 09:49:59 UTC) from {username='slave'} at 127.0.0.1:32806
I20240426 09:49:59.379920 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:59.380939 19118 tablet_replica.cc:330] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:59.381168 19118 raft_consensus.cc:2229] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:59.381436 19118 raft_consensus.cc:2258] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:59.382072 19118 ts_tablet_manager.cc:1875] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:59.393276 19119 tablet_replica.cc:330] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:49:59.393440 19119 raft_consensus.cc:2229] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Raft consensus shutting down.
I20240426 09:49:59.393638 19119 raft_consensus.cc:2258] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:59.393103 19117 tablet_replica.cc:330] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:49:59.394691 19117 raft_consensus.cc:2229] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:59.394838 19117 raft_consensus.cc:2258] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:59.395618 19117 ts_tablet_manager.cc:1875] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:59.399466 19117 ts_tablet_manager.cc:1888] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:59.399819 19117 log.cc:1196] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/bcdbd52812964134b82d9b2e4c31f60e
I20240426 09:49:59.400216 19117 ts_tablet_manager.cc:1909] T bcdbd52812964134b82d9b2e4c31f60e P 1aaf8befba984e199e96f53f46b50441: Deleting consensus metadata
I20240426 09:49:59.401086 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet bcdbd52812964134b82d9b2e4c31f60e (table rebalancer_dynamic_table_new_name [id=acb9f31f772647138524cd45b6f8cf27]) successfully deleted
I20240426 09:49:59.401517 19119 ts_tablet_manager.cc:1875] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240426 09:49:59.405675 19118 ts_tablet_manager.cc:1888] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:59.405754 19118 log.cc:1196] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/bcdbd52812964134b82d9b2e4c31f60e
I20240426 09:49:59.406302 19119 ts_tablet_manager.cc:1888] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240426 09:49:59.406394 19119 log.cc:1196] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/bcdbd52812964134b82d9b2e4c31f60e
I20240426 09:49:59.406678 19119 ts_tablet_manager.cc:1909] T bcdbd52812964134b82d9b2e4c31f60e P a692602e1a2e4794877e6fb92a7b9962: Deleting consensus metadata
I20240426 09:49:59.407358 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet bcdbd52812964134b82d9b2e4c31f60e (table rebalancer_dynamic_table_new_name [id=acb9f31f772647138524cd45b6f8cf27]) successfully deleted
I20240426 09:49:59.408012 19118 ts_tablet_manager.cc:1909] T bcdbd52812964134b82d9b2e4c31f60e P 3b41d87e7bfb419f8aa7500100129b6c: Deleting consensus metadata
I20240426 09:49:59.408620 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet bcdbd52812964134b82d9b2e4c31f60e (table rebalancer_dynamic_table_new_name [id=acb9f31f772647138524cd45b6f8cf27]) successfully deleted
I20240426 09:49:59.438513 9286 consensus_queue.cc:237] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:59.439678 6880 raft_consensus.cc:1261] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:59.439811 9669 raft_consensus.cc:1261] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:49:59.440172 19103 consensus_queue.cc:1035] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:59.440378 19091 consensus_queue.cc:1035] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:49:59.442174 17598 raft_consensus.cc:2932] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:59.442531 9669 raft_consensus.cc:2932] T aebef9b26fec446b8e71d4046768aedc P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:59.442600 6880 raft_consensus.cc:2932] T aebef9b26fec446b8e71d4046768aedc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:59.445780 6733 catalog_manager.cc:5484] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:59.450699 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet aebef9b26fec446b8e71d4046768aedc with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:49:59.450963 19118 tablet_replica.cc:330] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:49:59.451125 19118 raft_consensus.cc:2229] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:59.451272 19118 raft_consensus.cc:2258] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:59.451762 19118 ts_tablet_manager.cc:1875] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:59.453912 19118 ts_tablet_manager.cc:1888] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:49:59.453976 19118 log.cc:1196] T aebef9b26fec446b8e71d4046768aedc P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/aebef9b26fec446b8e71d4046768aedc
I20240426 09:49:59.454514 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet aebef9b26fec446b8e71d4046768aedc (table rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]) successfully deleted
I20240426 09:49:59.480199 6738 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240426 09:49:59.485729 9264 tablet_service.cc:1467] Processing CreateTablet for tablet 4da482df496340a0a41f7a4f3fed4021 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=80a6c7646ad5451bbdc500e7480a7ed5]), partition=
I20240426 09:49:59.485993 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 4da482df496340a0a41f7a4f3fed4021 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=80a6c7646ad5451bbdc500e7480a7ed5]), partition=
I20240426 09:49:59.486022 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4da482df496340a0a41f7a4f3fed4021. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.486214 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4da482df496340a0a41f7a4f3fed4021. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.488473 19086 tablet_bootstrap.cc:492] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:59.488515 19154 tablet_bootstrap.cc:492] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:59.488732 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 4da482df496340a0a41f7a4f3fed4021 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=80a6c7646ad5451bbdc500e7480a7ed5]), partition=
I20240426 09:49:59.488963 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4da482df496340a0a41f7a4f3fed4021. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.489440 19154 tablet_bootstrap.cc:654] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.489683 19086 tablet_bootstrap.cc:654] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.491318 18960 tablet_bootstrap.cc:492] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:59.492141 18960 tablet_bootstrap.cc:654] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.493393 19086 tablet_bootstrap.cc:492] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:59.493479 19086 ts_tablet_manager.cc:1367] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:59.494056 18960 tablet_bootstrap.cc:492] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:59.493980 19086 raft_consensus.cc:348] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:59.494129 19086 raft_consensus.cc:374] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.494144 18960 ts_tablet_manager.cc:1367] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:49:59.494170 19086 raft_consensus.cc:729] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.494287 19086 consensus_queue.cc:260] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:59.494591 19086 ts_tablet_manager.cc:1398] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:59.494568 18960 raft_consensus.cc:348] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:59.494652 18960 raft_consensus.cc:374] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.494673 18960 raft_consensus.cc:729] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.494742 18960 consensus_queue.cc:260] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:59.496766 18960 ts_tablet_manager.cc:1398] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:49:59.499343 19154 tablet_bootstrap.cc:492] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:59.499469 19154 ts_tablet_manager.cc:1367] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:49:59.500056 19154 raft_consensus.cc:348] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:59.500254 19154 raft_consensus.cc:374] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.500332 19154 raft_consensus.cc:729] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.500460 19154 consensus_queue.cc:260] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:59.500883 19154 ts_tablet_manager.cc:1398] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:59.563657 17387 raft_consensus.cc:482] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:59.563771 17387 raft_consensus.cc:504] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:59.564215 17387 leader_election.cc:288] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:59.564523 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4da482df496340a0a41f7a4f3fed4021" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:49:59.564679 9285 raft_consensus.cc:2454] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:59.564671 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4da482df496340a0a41f7a4f3fed4021" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:49:59.564823 7142 raft_consensus.cc:2454] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:59.565023 9483 leader_election.cc:302] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:59.565263 18995 raft_consensus.cc:2781] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:59.565332 18995 raft_consensus.cc:482] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:59.565394 18995 raft_consensus.cc:3037] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.566519 18995 raft_consensus.cc:504] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:59.566864 18995 leader_election.cc:288] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:49:59.567174 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4da482df496340a0a41f7a4f3fed4021" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:49:59.567314 9285 raft_consensus.cc:3037] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.568025 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4da482df496340a0a41f7a4f3fed4021" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:49:59.568150 7142 raft_consensus.cc:3037] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.568356 9285 raft_consensus.cc:2454] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:59.569180 7142 raft_consensus.cc:2454] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:59.569303 9483 leader_election.cc:302] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:59.569597 17387 raft_consensus.cc:2781] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:59.569696 17387 raft_consensus.cc:686] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:59.569828 17387 consensus_queue.cc:237] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:49:59.577733 6732 catalog_manager.cc:5484] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:49:59.653834 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:49:59.656355 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:49:59.665483 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:49:59.666800 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:49:59.667213 6993 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:49:59.668637 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:49:59.673177 9527 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:49:59.693387 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240426 09:49:59.696568 18212 consensus_queue.cc:1035] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.700330 17777 consensus_queue.cc:1035] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.703933 9396 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:49:59.708083 18212 consensus_queue.cc:1035] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.716929 18652 consensus_queue.cc:1035] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.728657 9285 raft_consensus.cc:1261] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:59.729060 17387 consensus_queue.cc:1035] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.730890 7143 raft_consensus.cc:1261] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:49:59.731559 18995 consensus_queue.cc:1035] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.733863 6732 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet 4da482df496340a0a41f7a4f3fed4021 (table rebalancer_dynamic_table [id=80a6c7646ad5451bbdc500e7480a7ed5]). Expected version 1 got 0
I20240426 09:49:59.735244 6732 catalog_manager.cc:5576] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021) does not have the latest schema for tablet 4da482df496340a0a41f7a4f3fed4021 (table rebalancer_dynamic_table [id=80a6c7646ad5451bbdc500e7480a7ed5]). Expected version 1 got 0
I20240426 09:49:59.746080 18871 tablet.cc:1722] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:59.746395 18871 tablet.cc:1620] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:59.747545 19016 tablet.cc:1722] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:59.747840 19016 tablet.cc:1620] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:59.752626 6736 catalog_manager.cc:5576] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545) does not have the latest schema for tablet 4da482df496340a0a41f7a4f3fed4021 (table rebalancer_dynamic_table [id=80a6c7646ad5451bbdc500e7480a7ed5]). Expected version 1 got 0
I20240426 09:49:59.754721 6738 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=80a6c7646ad5451bbdc500e7480a7ed5] alter complete (version 1)
I20240426 09:49:59.757273 19208 tablet.cc:1722] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240426 09:49:59.758790 19208 tablet.cc:1620] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:59.762516 18133 consensus_queue.cc:1035] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.765520 19205 consensus_queue.cc:1035] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:49:59.795142 9672 consensus_queue.cc:237] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:49:59.796444 7274 raft_consensus.cc:1261] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:59.796475 6880 raft_consensus.cc:1261] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:49:59.796864 17777 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:59.797040 18652 consensus_queue.cc:1035] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:49:59.798810 19230 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:59.799113 7274 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:59.799229 6880 raft_consensus.cc:2932] T 8e306113f9874277a60468d9248d45ce P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:59.802495 6736 catalog_manager.cc:5484] T 8e306113f9874277a60468d9248d45ce P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:49:59.807456 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e306113f9874277a60468d9248d45ce with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47690
I20240426 09:49:59.807817 19236 tablet_replica.cc:330] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:49:59.808032 19236 raft_consensus.cc:2229] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:49:59.808234 19236 raft_consensus.cc:2258] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:49:59.808715 19236 ts_tablet_manager.cc:1875] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:49:59.811264 19236 ts_tablet_manager.cc:1888] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:49:59.811388 19236 log.cc:1196] T 8e306113f9874277a60468d9248d45ce P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/8e306113f9874277a60468d9248d45ce
I20240426 09:49:59.811962 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 8e306113f9874277a60468d9248d45ce (table rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]) successfully deleted
I20240426 09:49:59.816867 6732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43346:
name: "rebalancer_extra_table_72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:49:59.822908 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 3c05f4899b1d441f8553f2dc970f6ed3 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.823233 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c05f4899b1d441f8553f2dc970f6ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.823207 7123 tablet_service.cc:1467] Processing CreateTablet for tablet 3c05f4899b1d441f8553f2dc970f6ed3 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.823473 7123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c05f4899b1d441f8553f2dc970f6ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.825323 7254 tablet_service.cc:1467] Processing CreateTablet for tablet 5b66a5a6cf3348ffbaa56679985e9aa2 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.825611 7254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b66a5a6cf3348ffbaa56679985e9aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.825721 18960 tablet_bootstrap.cc:492] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:49:59.825809 6862 tablet_service.cc:1467] Processing CreateTablet for tablet 96ba8066c09d403bb5299d25d6568a76 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.826069 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ba8066c09d403bb5299d25d6568a76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.827023 18960 tablet_bootstrap.cc:654] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.828187 9264 tablet_service.cc:1467] Processing CreateTablet for tablet 3c05f4899b1d441f8553f2dc970f6ed3 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.828258 19246 tablet_bootstrap.cc:492] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:49:59.828462 9264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c05f4899b1d441f8553f2dc970f6ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.829282 19246 tablet_bootstrap.cc:654] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.829689 9527 tablet_service.cc:1467] Processing CreateTablet for tablet 96ba8066c09d403bb5299d25d6568a76 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.829929 9527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ba8066c09d403bb5299d25d6568a76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.830230 19246 tablet_bootstrap.cc:492] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c: No bootstrap required, opened a new log
I20240426 09:49:59.830292 19246 ts_tablet_manager.cc:1367] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:59.830729 19246 raft_consensus.cc:348] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.830858 19246 raft_consensus.cc:374] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.830894 19246 raft_consensus.cc:729] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.830891 19154 tablet_bootstrap.cc:492] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:49:59.830996 19246 consensus_queue.cc:260] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.831038 6993 tablet_service.cc:1467] Processing CreateTablet for tablet 96ba8066c09d403bb5299d25d6568a76 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.831246 19246 ts_tablet_manager.cc:1398] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:59.831281 6993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ba8066c09d403bb5299d25d6568a76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.831764 19154 tablet_bootstrap.cc:654] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.833631 19249 tablet_bootstrap.cc:492] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:59.833834 6992 tablet_service.cc:1467] Processing CreateTablet for tablet 5b66a5a6cf3348ffbaa56679985e9aa2 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.834039 6992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b66a5a6cf3348ffbaa56679985e9aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.834786 18960 tablet_bootstrap.cc:492] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441: No bootstrap required, opened a new log
I20240426 09:49:59.834798 19249 tablet_bootstrap.cc:654] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.834877 18960 ts_tablet_manager.cc:1367] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:49:59.835355 18960 raft_consensus.cc:348] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:59.835441 18960 raft_consensus.cc:374] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.835461 18960 raft_consensus.cc:729] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.835548 18960 consensus_queue.cc:260] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:59.835757 18960 ts_tablet_manager.cc:1398] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:59.838514 19086 tablet_bootstrap.cc:492] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:49:59.839577 19086 tablet_bootstrap.cc:654] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.840325 9396 tablet_service.cc:1467] Processing CreateTablet for tablet 5b66a5a6cf3348ffbaa56679985e9aa2 (DEFAULT_TABLE table=rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:49:59.840641 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b66a5a6cf3348ffbaa56679985e9aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:49:59.843541 19243 tablet_bootstrap.cc:492] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:59.844573 19243 tablet_bootstrap.cc:654] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.845714 19249 tablet_bootstrap.cc:492] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:59.845810 19249 ts_tablet_manager.cc:1367] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240426 09:49:59.845820 19243 tablet_bootstrap.cc:492] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:59.845899 19243 ts_tablet_manager.cc:1367] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:49:59.846266 19249 raft_consensus.cc:348] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.846371 19249 raft_consensus.cc:374] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.846401 19249 raft_consensus.cc:729] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.846328 19243 raft_consensus.cc:348] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:59.846441 19243 raft_consensus.cc:374] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.846474 19243 raft_consensus.cc:729] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.846503 19249 consensus_queue.cc:260] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.846588 19243 consensus_queue.cc:260] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:59.846812 19249 ts_tablet_manager.cc:1398] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:49:59.846907 19243 ts_tablet_manager.cc:1398] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:59.846975 19249 tablet_bootstrap.cc:492] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:49:59.848591 19243 tablet_bootstrap.cc:492] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:49:59.849730 19249 tablet_bootstrap.cc:654] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.851799 19249 tablet_bootstrap.cc:492] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899: No bootstrap required, opened a new log
I20240426 09:49:59.851882 19249 ts_tablet_manager.cc:1367] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:49:59.852295 19249 raft_consensus.cc:348] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.852427 19249 raft_consensus.cc:374] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.852469 19249 raft_consensus.cc:729] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.852579 19249 consensus_queue.cc:260] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.852870 19249 ts_tablet_manager.cc:1398] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:49:59.854483 19154 tablet_bootstrap.cc:492] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962: No bootstrap required, opened a new log
I20240426 09:49:59.854585 19154 ts_tablet_manager.cc:1367] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20240426 09:49:59.855108 19154 raft_consensus.cc:348] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:59.855262 19154 raft_consensus.cc:374] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.855309 19154 raft_consensus.cc:729] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.855366 19243 tablet_bootstrap.cc:654] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.855456 19154 consensus_queue.cc:260] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:49:59.855799 19154 ts_tablet_manager.cc:1398] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:59.856104 6732 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43346:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240426 09:49:59.857622 19245 tablet_bootstrap.cc:492] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:49:59.858952 19243 tablet_bootstrap.cc:492] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: No bootstrap required, opened a new log
I20240426 09:49:59.859046 19243 ts_tablet_manager.cc:1367] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240426 09:49:59.859223 19086 tablet_bootstrap.cc:492] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: No bootstrap required, opened a new log
I20240426 09:49:59.859306 19086 ts_tablet_manager.cc:1367] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240426 09:49:59.859593 19243 raft_consensus.cc:348] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.859704 19243 raft_consensus.cc:374] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.859735 19243 raft_consensus.cc:729] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.859812 19086 raft_consensus.cc:348] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.859869 19243 consensus_queue.cc:260] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.859910 19086 raft_consensus.cc:374] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.859944 19086 raft_consensus.cc:729] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.860056 19086 consensus_queue.cc:260] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.860190 19243 ts_tablet_manager.cc:1398] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:59.860332 19086 ts_tablet_manager.cc:1398] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:59.864915 19245 tablet_bootstrap.cc:654] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff: Neither blocks nor log segments found. Creating new log.
I20240426 09:49:59.866935 19208 tablet.cc:1722] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:59.867239 19208 tablet.cc:1620] T 4da482df496340a0a41f7a4f3fed4021 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: MemRowSet was empty: no flush needed.
I20240426 09:49:59.867789 19016 tablet.cc:1722] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:59.868072 19016 tablet.cc:1620] T 4da482df496340a0a41f7a4f3fed4021 P a692602e1a2e4794877e6fb92a7b9962: MemRowSet was empty: no flush needed.
I20240426 09:49:59.869679 18871 tablet.cc:1722] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240426 09:49:59.869938 18871 tablet.cc:1620] T 4da482df496340a0a41f7a4f3fed4021 P 1aaf8befba984e199e96f53f46b50441: MemRowSet was empty: no flush needed.
I20240426 09:49:59.873070 19245 tablet_bootstrap.cc:492] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff: No bootstrap required, opened a new log
I20240426 09:49:59.873171 19245 ts_tablet_manager.cc:1367] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240426 09:49:59.873587 19245 raft_consensus.cc:348] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.873703 19245 raft_consensus.cc:374] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:49:59.873744 19245 raft_consensus.cc:729] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:49:59.873872 19245 consensus_queue.cc:260] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.874230 19245 ts_tablet_manager.cc:1398] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:49:59.879844 6736 catalog_manager.cc:6029] T 00000000000000000000000000000000 P 27345b30edb745aa91af53b9ce01d7da: rebalancer_dynamic_table [id=80a6c7646ad5451bbdc500e7480a7ed5] alter complete (version 2)
I20240426 09:49:59.931988 17387 raft_consensus.cc:482] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:59.932107 17387 raft_consensus.cc:504] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.932512 17387 leader_election.cc:288] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:59.932824 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ba8066c09d403bb5299d25d6568a76" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c" is_pre_election: true
I20240426 09:49:59.932971 6880 raft_consensus.cc:2454] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:59.933056 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ba8066c09d403bb5299d25d6568a76" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:59.933211 7011 raft_consensus.cc:2454] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 0.
I20240426 09:49:59.933495 9484 leader_election.cc:302] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8e863bc0edf4940bbf3e9f9b2bbb3e5, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:49:59.933787 17387 raft_consensus.cc:2781] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:59.933858 17387 raft_consensus.cc:482] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:59.933903 17387 raft_consensus.cc:3037] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.940517 17387 raft_consensus.cc:504] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.940950 17387 leader_election.cc:288] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:49:59.941262 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ba8066c09d403bb5299d25d6568a76" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:59.941401 7011 raft_consensus.cc:3037] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.941602 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ba8066c09d403bb5299d25d6568a76" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:49:59.941740 6880 raft_consensus.cc:3037] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.942564 7011 raft_consensus.cc:2454] T 96ba8066c09d403bb5299d25d6568a76 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:59.942855 6880 raft_consensus.cc:2454] T 96ba8066c09d403bb5299d25d6568a76 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 1.
I20240426 09:49:59.942934 9482 leader_election.cc:302] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:49:59.943171 17387 raft_consensus.cc:2781] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:59.943259 17387 raft_consensus.cc:686] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:49:59.943392 17387 consensus_queue.cc:237] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:49:59.945844 18872 raft_consensus.cc:482] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:49:59.945961 18872 raft_consensus.cc:504] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.946338 18872 leader_election.cc:288] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:59.946913 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b66a5a6cf3348ffbaa56679985e9aa2" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" is_pre_election: true
I20240426 09:49:59.946835 6731 catalog_manager.cc:5484] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:49:59.947054 7274 raft_consensus.cc:2454] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:59.947432 9352 leader_election.cc:302] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:59.948232 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b66a5a6cf3348ffbaa56679985e9aa2" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" is_pre_election: true
I20240426 09:49:59.948374 7011 raft_consensus.cc:2454] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:49:59.948381 18872 raft_consensus.cc:2781] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:49:59.948441 18872 raft_consensus.cc:482] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:49:59.948482 18872 raft_consensus.cc:3037] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.949596 18872 raft_consensus.cc:504] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.949944 18872 leader_election.cc:288] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:49:59.950389 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b66a5a6cf3348ffbaa56679985e9aa2" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:49:59.950521 7274 raft_consensus.cc:3037] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.951617 7274 raft_consensus.cc:2454] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:59.952044 9352 leader_election.cc:302] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:49:59.952731 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b66a5a6cf3348ffbaa56679985e9aa2" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:49:59.952852 7011 raft_consensus.cc:3037] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:49:59.953929 7011 raft_consensus.cc:2454] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:49:59.953975 19286 raft_consensus.cc:2781] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:49:59.954053 19286 raft_consensus.cc:686] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:49:59.954203 19286 consensus_queue.cc:237] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:49:59.966781 6732 catalog_manager.cc:5484] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:00.078822 19288 raft_consensus.cc:482] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:00.078940 19288 raft_consensus.cc:504] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:00.079329 19288 leader_election.cc:288] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:00.079653 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c05f4899b1d441f8553f2dc970f6ed3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962" is_pre_election: true
I20240426 09:50:00.079659 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c05f4899b1d441f8553f2dc970f6ed3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441" is_pre_election: true
I20240426 09:50:00.079782 9285 raft_consensus.cc:2454] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:50:00.079782 7142 raft_consensus.cc:2454] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 0.
I20240426 09:50:00.080032 9351 leader_election.cc:302] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:50:00.080201 19288 raft_consensus.cc:2781] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:00.080261 19288 raft_consensus.cc:482] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:00.080317 19288 raft_consensus.cc:3037] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:00.081389 19288 raft_consensus.cc:504] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:00.081732 19288 leader_election.cc:288] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:00.081960 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c05f4899b1d441f8553f2dc970f6ed3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:50:00.081975 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c05f4899b1d441f8553f2dc970f6ed3" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:50:00.082044 9285 raft_consensus.cc:3037] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:00.082094 7142 raft_consensus.cc:3037] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:00.083066 7142 raft_consensus.cc:2454] T 3c05f4899b1d441f8553f2dc970f6ed3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:50:00.083389 9350 leader_election.cc:302] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 3b41d87e7bfb419f8aa7500100129b6c; no voters:
I20240426 09:50:00.083395 9285 raft_consensus.cc:2454] T 3c05f4899b1d441f8553f2dc970f6ed3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 1.
I20240426 09:50:00.083554 19288 raft_consensus.cc:2781] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:00.083628 19288 raft_consensus.cc:686] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:50:00.083837 19288 consensus_queue.cc:237] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:00.086135 6738 catalog_manager.cc:5484] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:00.120116 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:00.121979 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.122123 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.122742 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:00.128883 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:00.134235 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:00.156684 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:50:00.182638 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.233274 7011 consensus_queue.cc:237] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:00.234519 6880 raft_consensus.cc:1261] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:00.234910 9285 raft_consensus.cc:1261] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:00.235010 19011 consensus_queue.cc:1035] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:00.235285 18261 consensus_queue.cc:1035] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:00.236893 19011 raft_consensus.cc:2932] T 4e3624733f3444778b8656b3289d57dc P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:00.237519 6881 raft_consensus.cc:2932] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:00.238435 9285 raft_consensus.cc:2932] T 4e3624733f3444778b8656b3289d57dc P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:00.241225 6738 catalog_manager.cc:5484] T 4e3624733f3444778b8656b3289d57dc P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:00.244567 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3624733f3444778b8656b3289d57dc with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:00.244802 19236 tablet_replica.cc:330] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:00.244956 19236 raft_consensus.cc:2229] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:00.245071 19236 raft_consensus.cc:2258] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:00.245533 19236 ts_tablet_manager.cc:1875] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:00.247768 19236 ts_tablet_manager.cc:1888] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:00.248466 19236 log.cc:1196] T 4e3624733f3444778b8656b3289d57dc P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/4e3624733f3444778b8656b3289d57dc
I20240426 09:50:00.249171 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 4e3624733f3444778b8656b3289d57dc (table rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]) successfully deleted
I20240426 09:50:00.368276 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:50:00.370939 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:00.372131 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.384806 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:00.395213 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:00.406888 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.407071 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.415464 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:00.451123 19288 consensus_queue.cc:1035] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:00.461190 18872 consensus_queue.cc:1035] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:00.487149 9669 consensus_queue.cc:237] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:00.488430 9546 raft_consensus.cc:1261] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:00.488433 7142 raft_consensus.cc:1261] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:00.488751 19230 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:00.489015 18652 consensus_queue.cc:1035] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:00.490820 18652 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:00.491148 7142 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:00.494297 9546 raft_consensus.cc:2932] T bbdcd6105a2647738257638e1e1096c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:00.494817 6737 catalog_manager.cc:5484] T bbdcd6105a2647738257638e1e1096c3 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:00.499115 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet bbdcd6105a2647738257638e1e1096c3 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:00.499527 19401 tablet_replica.cc:330] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:00.499730 19401 raft_consensus.cc:2229] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:00.500002 19401 raft_consensus.cc:2258] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:00.500556 19401 ts_tablet_manager.cc:1875] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:00.503448 19401 ts_tablet_manager.cc:1888] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:00.503605 19401 log.cc:1196] T bbdcd6105a2647738257638e1e1096c3 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/bbdcd6105a2647738257638e1e1096c3
I20240426 09:50:00.504160 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet bbdcd6105a2647738257638e1e1096c3 (table rebalancer_extra_table_4 [id=7ee079088cce4790a17ade04fbcd9743]) successfully deleted
I20240426 09:50:00.540449 18995 consensus_queue.cc:1035] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:00.544271 17387 consensus_queue.cc:1035] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:00.631381 18872 consensus_queue.cc:1035] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:00.646327 19288 consensus_queue.cc:1035] T 3c05f4899b1d441f8553f2dc970f6ed3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:00.667940 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:00.673825 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:00.676411 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.679718 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:50:00.685571 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.711696 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:00.712327 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.727110 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:00.782164 6881 consensus_queue.cc:237] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:00.783499 9546 raft_consensus.cc:1261] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:00.783530 7011 raft_consensus.cc:1261] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:00.784004 18560 consensus_queue.cc:1035] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:00.784360 18994 consensus_queue.cc:1035] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:00.786089 18560 raft_consensus.cc:2932] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:00.786518 7011 raft_consensus.cc:2932] T 3f8803aa38a646ebbdf5018ab7f65845 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:00.790241 6737 catalog_manager.cc:5484] T 3f8803aa38a646ebbdf5018ab7f65845 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:00.791440 9546 raft_consensus.cc:2932] T 3f8803aa38a646ebbdf5018ab7f65845 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:00.794945 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f8803aa38a646ebbdf5018ab7f65845 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:00.795356 19480 tablet_replica.cc:330] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:00.795912 19480 raft_consensus.cc:2229] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:00.796070 19480 raft_consensus.cc:2258] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:00.796561 19480 ts_tablet_manager.cc:1875] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:00.799491 19480 ts_tablet_manager.cc:1888] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:00.799635 19480 log.cc:1196] T 3f8803aa38a646ebbdf5018ab7f65845 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/3f8803aa38a646ebbdf5018ab7f65845
I20240426 09:50:00.800240 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 3f8803aa38a646ebbdf5018ab7f65845 (table rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]) successfully deleted
I20240426 09:50:00.941604 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:00.946831 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:00.948364 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.950191 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.966334 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:00.967681 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:00.971251 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:00.974985 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:50:01.055089 9285 consensus_queue.cc:237] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:01.056351 9549 raft_consensus.cc:1261] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:01.056792 19517 consensus_queue.cc:1035] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:01.056951 9669 raft_consensus.cc:1261] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:01.057353 19517 consensus_queue.cc:1035] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:01.058583 19091 raft_consensus.cc:2932] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:01.059063 9549 raft_consensus.cc:2932] T 638b15a2c9ea42c3b86a985ccae550a7 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:01.059942 9669 raft_consensus.cc:2932] T 638b15a2c9ea42c3b86a985ccae550a7 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:01.063733 6738 catalog_manager.cc:5484] T 638b15a2c9ea42c3b86a985ccae550a7 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:01.066581 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 638b15a2c9ea42c3b86a985ccae550a7 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:01.068231 19548 tablet_replica.cc:330] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:01.068375 19548 raft_consensus.cc:2229] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:01.070102 19548 raft_consensus.cc:2258] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:01.070616 19548 ts_tablet_manager.cc:1875] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:01.072983 19548 ts_tablet_manager.cc:1888] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:01.073064 19548 log.cc:1196] T 638b15a2c9ea42c3b86a985ccae550a7 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/638b15a2c9ea42c3b86a985ccae550a7
I20240426 09:50:01.073832 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 638b15a2c9ea42c3b86a985ccae550a7 (table rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]) successfully deleted
I20240426 09:50:01.196069 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.210347 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:01.210130 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:01.214157 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:01.217168 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:50:01.231559 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:01.254462 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.254918 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.294623 9669 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa"
dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39572
I20240426 09:50:01.294924 9669 raft_consensus.cc:595] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:01.416277 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:01.425276 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.425491 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:50:01.427390 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:01.436082 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:01.450865 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.457345 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.460744 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:01.621225 19230 raft_consensus.cc:979] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: : Instructing follower c244e02f3cf847c0a5b69e1059f1071c to start an election
I20240426 09:50:01.621362 19230 raft_consensus.cc:1067] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Signalling peer c244e02f3cf847c0a5b69e1059f1071c to start an election
I20240426 09:50:01.622583 6881 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa"
dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
from {username='slave'} at 127.0.99.136:56221
I20240426 09:50:01.622802 6881 raft_consensus.cc:482] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:01.622881 6881 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:01.624061 6881 raft_consensus.cc:504] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:01.624540 6881 leader_election.cc:288] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 2 election: Requested vote from peers 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059), 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:01.625051 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:50:01.625171 9285 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:01.626189 9285 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 2.
I20240426 09:50:01.626457 9416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:50:01.626585 9416 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:01.626924 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "c244e02f3cf847c0a5b69e1059f1071c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:50:01.627044 9669 raft_consensus.cc:3032] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:01.627111 9669 raft_consensus.cc:729] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:50:01.627267 9669 consensus_queue.cc:260] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:01.627476 9669 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:01.628504 9669 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 2.
I20240426 09:50:01.630738 9416 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c244e02f3cf847c0a5b69e1059f1071c in term 2.
I20240426 09:50:01.630894 6815 leader_election.cc:302] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:50:01.631076 19642 raft_consensus.cc:2781] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:01.631171 19642 raft_consensus.cc:686] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Becoming Leader. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:50:01.631315 19642 consensus_queue.cc:237] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:01.634502 6734 catalog_manager.cc:5484] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: term changed from 1 to 2, leader changed from 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) to c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129). New cstate: current_term: 2 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:01.669298 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:01.678611 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:01.685609 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.687803 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:01.692907 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.693145 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:01.694911 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:01.711091 9264 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240426 09:50:01.765785 7274 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4189112964fb425aac074352dd873415"
dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:58038
I20240426 09:50:01.765923 7274 raft_consensus.cc:595] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:01.771658 18212 raft_consensus.cc:979] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: : Instructing follower a692602e1a2e4794877e6fb92a7b9962 to start an election
I20240426 09:50:01.771781 18212 raft_consensus.cc:1067] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Signalling peer a692602e1a2e4794877e6fb92a7b9962 to start an election
I20240426 09:50:01.772236 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4189112964fb425aac074352dd873415"
dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
from {username='slave'} at 127.0.99.132:38007
I20240426 09:50:01.772426 9286 raft_consensus.cc:482] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:01.772506 9286 raft_consensus.cc:3037] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:01.773661 9286 raft_consensus.cc:504] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:01.774070 9286 leader_election.cc:288] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 2 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:01.774394 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4189112964fb425aac074352dd873415" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:50:01.774518 6881 raft_consensus.cc:3037] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:01.774976 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4189112964fb425aac074352dd873415" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:50:01.775103 9669 raft_consensus.cc:3037] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:01.775074 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4189112964fb425aac074352dd873415" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:50:01.775192 7275 raft_consensus.cc:3032] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:01.775236 7275 raft_consensus.cc:729] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:50:01.775419 7275 consensus_queue.cc:260] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:01.775559 7275 raft_consensus.cc:3037] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:01.775959 9669 raft_consensus.cc:2454] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 2.
I20240426 09:50:01.776263 7275 raft_consensus.cc:2454] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 2.
I20240426 09:50:01.776556 9221 leader_election.cc:302] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:50:01.776733 19701 raft_consensus.cc:2781] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:01.776814 19701 raft_consensus.cc:686] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [term 2 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:50:01.776942 19701 consensus_queue.cc:237] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:01.777438 6881 raft_consensus.cc:2454] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 2.
I20240426 09:50:01.779557 6737 catalog_manager.cc:5484] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 1 to 2, leader changed from e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 2 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:01.904902 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:01.906302 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.906345 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:01.918491 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:01.920039 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:01.920850 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:01.935930 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:01.962739 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:02.053571 9669 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:02.054039 19642 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:02.120048 9416 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:02.120553 19642 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:02.130548 9285 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:02.130991 19642 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:02.140080 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:02.140499 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:02.143646 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:02.144565 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:02.153740 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:02.154206 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:02.164680 6881 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:02.165061 19701 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:02.180465 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:02.193173 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:02.204892 9669 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:02.205364 19517 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:02.225284 7275 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:02.226045 19701 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:02.254431 9285 consensus_queue.cc:237] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:02.255635 6881 raft_consensus.cc:1261] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:02.255780 9417 raft_consensus.cc:1261] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:02.256088 19701 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:02.256405 19829 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:02.257972 19829 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:02.258520 9416 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:02.259245 6881 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:02.261443 6739 catalog_manager.cc:5484] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:02.264395 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet ab6450f48a36483883f8914075c224a8 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:02.266027 19850 tablet_replica.cc:330] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:02.266145 19850 raft_consensus.cc:2229] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:02.266235 19850 raft_consensus.cc:2258] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:02.266605 19850 ts_tablet_manager.cc:1875] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:02.268697 19850 ts_tablet_manager.cc:1888] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:02.268764 19850 log.cc:1196] T ab6450f48a36483883f8914075c224a8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/ab6450f48a36483883f8914075c224a8
I20240426 09:50:02.269253 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet ab6450f48a36483883f8914075c224a8 (table rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]) successfully deleted
I20240426 09:50:02.614490 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:02.620585 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:02.631552 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:02.634663 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:02.644877 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:02.645774 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:02.655581 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:02.674181 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:02.729990 7274 consensus_queue.cc:237] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:02.731241 9416 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:02.731333 9285 raft_consensus.cc:1261] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:02.731702 18705 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:02.731885 18212 consensus_queue.cc:1035] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:02.733619 19938 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:02.733945 9285 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:02.733945 9416 raft_consensus.cc:2932] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:02.737457 6737 catalog_manager.cc:5484] T b2d42c52b7a94e32ab4cdad78ce884ee P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 6 to 7, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:02.740289 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d42c52b7a94e32ab4cdad78ce884ee with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47654
I20240426 09:50:02.741050 19939 tablet_replica.cc:330] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:50:02.741233 19939 raft_consensus.cc:2229] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:02.741554 19939 raft_consensus.cc:2258] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:02.742457 19939 ts_tablet_manager.cc:1875] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:02.744910 19939 ts_tablet_manager.cc:1888] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240426 09:50:02.744977 19939 log.cc:1196] T b2d42c52b7a94e32ab4cdad78ce884ee P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/b2d42c52b7a94e32ab4cdad78ce884ee
I20240426 09:50:02.745508 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet b2d42c52b7a94e32ab4cdad78ce884ee (table rebalance_test_table_1 [id=a49eca66f5a942d0857531f4e24e59f4]) successfully deleted
I20240426 09:50:02.866278 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:02.871311 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:02.872759 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:02.873364 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:02.876359 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:02.889570 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:02.915508 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:02.934708 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:02.991250 7274 consensus_queue.cc:237] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:02.992583 7142 raft_consensus.cc:1261] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:02.992995 9669 raft_consensus.cc:1261] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:02.993094 18212 consensus_queue.cc:1035] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:02.993443 19938 consensus_queue.cc:1035] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:02.995126 18212 raft_consensus.cc:2932] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:02.999166 6737 catalog_manager.cc:5484] T d83c01e6dc6d4b5b9365243058f459a1 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:02.999519 7142 raft_consensus.cc:2932] T d83c01e6dc6d4b5b9365243058f459a1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:03.002425 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet d83c01e6dc6d4b5b9365243058f459a1 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:03.004163 9669 raft_consensus.cc:2932] T d83c01e6dc6d4b5b9365243058f459a1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:03.004660 20015 tablet_replica.cc:330] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:03.004863 20015 raft_consensus.cc:2229] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:03.005008 20015 raft_consensus.cc:2258] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:03.005403 20015 ts_tablet_manager.cc:1875] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:03.009791 20015 ts_tablet_manager.cc:1888] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:03.009882 20015 log.cc:1196] T d83c01e6dc6d4b5b9365243058f459a1 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/d83c01e6dc6d4b5b9365243058f459a1
I20240426 09:50:03.010425 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet d83c01e6dc6d4b5b9365243058f459a1 (table rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]) successfully deleted
I20240426 09:50:03.127390 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:03.135082 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:03.138731 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:03.139755 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:03.140717 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:03.153569 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:03.153898 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:03.177659 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:03.238490 9546 consensus_queue.cc:237] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:03.239894 7013 raft_consensus.cc:1261] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:50:03.239888 7274 raft_consensus.cc:1261] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:50:03.240247 19919 consensus_queue.cc:1035] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:03.240394 18995 consensus_queue.cc:1035] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:03.242198 19919 raft_consensus.cc:2932] T 0d04a7ec2c62437b879de204b5c885e9 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:03.242512 7275 raft_consensus.cc:2932] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:03.243284 7013 raft_consensus.cc:2932] T 0d04a7ec2c62437b879de204b5c885e9 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:03.245916 6731 catalog_manager.cc:5484] T 0d04a7ec2c62437b879de204b5c885e9 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 4, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:03.249414 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d04a7ec2c62437b879de204b5c885e9 with delete_type TABLET_DATA_TOMBSTONED (TS c244e02f3cf847c0a5b69e1059f1071c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:44196
I20240426 09:50:03.250164 20089 tablet_replica.cc:330] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:50:03.250355 20089 raft_consensus.cc:2229] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:03.250707 20089 raft_consensus.cc:2258] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:03.251166 20089 ts_tablet_manager.cc:1875] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:03.253952 20089 ts_tablet_manager.cc:1888] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240426 09:50:03.254040 20089 log.cc:1196] T 0d04a7ec2c62437b879de204b5c885e9 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/0d04a7ec2c62437b879de204b5c885e9
I20240426 09:50:03.254601 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 0d04a7ec2c62437b879de204b5c885e9 (table rebalancer_extra_table_48 [id=7c4113493b4b4de8bc21acb6011e65da]) successfully deleted
I20240426 09:50:03.456543 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:03.463344 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:03.467201 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:03.475373 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:03.478174 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:03.478632 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:03.486630 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:03.489638 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:03.570657 6880 consensus_queue.cc:237] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:03.572268 9285 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:50:03.572455 9417 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:50:03.572894 19822 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:03.573098 18994 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:03.575546 19822 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:03.575930 9417 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:03.579402 6739 catalog_manager.cc:5484] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 2 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:03.583245 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 2770cd77ebb642e3a52183a3871e33fa with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:03.584873 9285 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:03.585719 20224 tablet_replica.cc:330] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:03.585935 20224 raft_consensus.cc:2229] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:03.589488 20224 raft_consensus.cc:2258] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:03.590549 20224 ts_tablet_manager.cc:1875] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:03.594290 20224 ts_tablet_manager.cc:1888] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240426 09:50:03.594437 20224 log.cc:1196] T 2770cd77ebb642e3a52183a3871e33fa P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/2770cd77ebb642e3a52183a3871e33fa
I20240426 09:50:03.598930 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 2770cd77ebb642e3a52183a3871e33fa (table rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]) successfully deleted
I20240426 09:50:03.723701 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:03.731382 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:03.735983 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:03.744760 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:03.754021 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:03.754611 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:03.759184 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:03.772146 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:03.833146 9285 consensus_queue.cc:237] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:03.834455 9672 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240426 09:50:03.834465 6880 raft_consensus.cc:1261] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240426 09:50:03.834901 19829 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240426 09:50:03.835124 19701 consensus_queue.cc:1035] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240426 09:50:03.836807 19701 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:03.837265 6880 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:03.840975 9672 raft_consensus.cc:2932] T 4189112964fb425aac074352dd873415 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:03.841990 6731 catalog_manager.cc:5484] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 6 to 8, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 2 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:03.845089 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 4189112964fb425aac074352dd873415 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:03.845469 20300 tablet_replica.cc:330] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:03.845651 20300 raft_consensus.cc:2229] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:03.845826 20300 raft_consensus.cc:2258] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:03.846386 20300 ts_tablet_manager.cc:1875] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:03.848779 20300 ts_tablet_manager.cc:1888] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20240426 09:50:03.848848 20300 log.cc:1196] T 4189112964fb425aac074352dd873415 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/4189112964fb425aac074352dd873415
I20240426 09:50:03.849587 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 4189112964fb425aac074352dd873415 (table rebalance_test_table_17 [id=73fbba72847f467b9f7b718c165c2c5f]) successfully deleted
I20240426 09:50:03.868824 7274 consensus_queue.cc:237] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:03.870545 6880 raft_consensus.cc:1261] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:03.870968 20317 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:50:03.871484 7208 consensus_peers.cc:485] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 722306427b7b46b4b0942193c821d990. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:03.871747 9546 raft_consensus.cc:1261] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:03.872257 20317 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:03.875587 18705 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:03.875954 6880 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:03.877847 9546 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:03.879642 6733 catalog_manager.cc:5484] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:03.904314 6880 consensus_queue.cc:237] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:03.905647 9285 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240426 09:50:03.905844 6815 consensus_peers.cc:485] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 948e1df8d52f42cebcd289fff377d6fb. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:03.906055 20337 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:03.906291 7274 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:03.906620 20337 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:03.908341 19822 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:03.908735 7274 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:03.910553 9285 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:03.912357 6731 catalog_manager.cc:5484] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:03.951896 6880 consensus_queue.cc:237] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:03.953445 7274 raft_consensus.cc:1261] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:03.953883 7013 raft_consensus.cc:1261] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:03.954283 20340 consensus_queue.cc:1035] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:03.954847 20340 consensus_queue.cc:1035] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:03.955044 6815 consensus_peers.cc:485] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b967d83e93d843398e93077285865b1b. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:03.956564 18994 raft_consensus.cc:2932] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:03.958142 7011 raft_consensus.cc:2932] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:03.958318 7274 raft_consensus.cc:2932] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:03.962791 6731 catalog_manager.cc:5484] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:03.989404 9672 consensus_queue.cc:237] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:03.990892 7274 raft_consensus.cc:1261] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240426 09:50:03.990931 6880 raft_consensus.cc:1261] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240426 09:50:03.991345 20274 consensus_queue.cc:1035] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:03.991662 18652 consensus_queue.cc:1035] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:03.991815 9614 consensus_peers.cc:485] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca0a2b121aba4659bf2d3a475a1e298b. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:03.993827 19992 raft_consensus.cc:2932] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:03.994256 7274 raft_consensus.cc:2932] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:03.996676 6880 raft_consensus.cc:2932] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:03.997309 6733 catalog_manager.cc:5484] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 2 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:04.018175 7142 consensus_queue.cc:237] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:04.019474 9417 raft_consensus.cc:1261] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:04.019584 6880 raft_consensus.cc:1261] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:04.019728 7077 consensus_peers.cc:485] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d720846739e49f888da92befe06a309. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:04.019907 20394 consensus_queue.cc:1035] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.020254 19205 consensus_queue.cc:1035] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.022213 19205 raft_consensus.cc:2932] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.024144 9417 raft_consensus.cc:2932] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.024118 6880 raft_consensus.cc:2932] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.027603 6737 catalog_manager.cc:5484] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.047818 9416 consensus_queue.cc:237] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:04.049343 9672 raft_consensus.cc:1261] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:04.049635 7274 raft_consensus.cc:1261] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:04.049808 20413 consensus_queue.cc:1035] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.050069 20232 consensus_queue.cc:1035] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:04.050256 9352 consensus_peers.cc:485] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6dce6f1e165c492ab23022527bac580a. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:04.052096 20232 raft_consensus.cc:2932] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:04.052469 9672 raft_consensus.cc:2932] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:04.055774 6731 catalog_manager.cc:5484] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:04.055816 7274 raft_consensus.cc:2932] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:04.077672 7142 consensus_queue.cc:237] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:04.079188 9546 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:04.079202 9285 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:04.079314 7077 consensus_peers.cc:485] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 838bef0a35c046e39b465f2ea743350f. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:04.079562 19205 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.079731 20394 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.081790 19205 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:04.082176 9285 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:04.086499 9546 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:04.086993 6735 catalog_manager.cc:5484] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:04.107462 7011 consensus_queue.cc:237] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
W20240426 09:50:04.109082 6946 consensus_peers.cc:485] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c94c820ab65420a8d3f885ccc9f8884. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:04.109148 9416 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:04.109508 20070 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:04.109781 7142 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:04.110263 20070 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:04.112043 19011 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.112454 7142 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.114912 9416 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.115422 6733 catalog_manager.cc:5484] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 5 to 6, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.141366 9285 consensus_queue.cc:237] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:04.143294 9546 raft_consensus.cc:1261] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:04.143406 9219 consensus_peers.cc:485] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6cb4534d05c49ed859b4abe61097ac8. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:04.143668 9669 raft_consensus.cc:1261] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:04.145196 19829 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.145524 20294 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.147933 19829 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:04.148352 9669 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:04.152179 6735 catalog_manager.cc:5484] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:04.153949 9546 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:04.182507 9546 consensus_queue.cc:237] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:04.183996 7274 raft_consensus.cc:1261] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:04.184294 9484 consensus_peers.cc:485] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11604d6436a946eba63a36c28e1b0290. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:04.184352 7011 raft_consensus.cc:1261] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:04.184536 20489 consensus_queue.cc:1035] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.184772 19919 consensus_queue.cc:1035] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.186967 20489 raft_consensus.cc:2932] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:04.187367 7274 raft_consensus.cc:2932] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:04.190739 6731 catalog_manager.cc:5484] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:04.192142 7011 raft_consensus.cc:2932] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:04.215642 9416 consensus_queue.cc:237] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:04.217018 9285 raft_consensus.cc:1261] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:04.217103 9350 consensus_peers.cc:485] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d2a5223682934aa698dd23aa6eab941b. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:04.217372 20416 consensus_queue.cc:1035] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.217729 6880 raft_consensus.cc:1261] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:04.218071 20416 consensus_queue.cc:1035] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:04.219830 20416 raft_consensus.cc:2932] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:04.220197 6880 raft_consensus.cc:2932] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:04.221745 9285 raft_consensus.cc:2932] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:04.225476 6735 catalog_manager.cc:5484] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:04.246034 7143 consensus_queue.cc:237] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:04.247381 9285 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:04.247627 7274 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:04.247800 20435 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:04.248054 20394 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20240426 09:50:04.248267 7077 consensus_peers.cc:485] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ba744a156204d9e81db6ea113945203. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:04.249688 20435 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.250999 7274 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.251713 9285 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.253600 6731 catalog_manager.cc:5484] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 5 to 6, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:04.268501 20538 ts_tablet_manager.cc:902] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:50:04.269049 20538 tablet_copy_client.cc:308] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:50:04.269598 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 722306427b7b46b4b0942193c821d990 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:56047)
I20240426 09:50:04.269706 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 722306427b7b46b4b0942193c821d990 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:56047: session id = 1aaf8befba984e199e96f53f46b50441-722306427b7b46b4b0942193c821d990
I20240426 09:50:04.270732 7295 tablet_copy_source_session.cc:205] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.271662 20538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722306427b7b46b4b0942193c821d990. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.274052 20538 tablet_copy_client.cc:783] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.274209 20538 tablet_copy_client.cc:647] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.275668 20538 tablet_copy_client.cc:523] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.276703 7143 consensus_queue.cc:237] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:04.277015 20538 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:04.278209 7274 raft_consensus.cc:1261] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:04.278405 9546 raft_consensus.cc:1261] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:04.278496 7077 consensus_peers.cc:485] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:04.278666 20394 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:04.278854 20435 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:04.280640 20394 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:04.281069 9546 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:04.281306 20538 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.281548 20538 tablet_bootstrap.cc:492] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:04.281693 20538 ts_tablet_manager.cc:1367] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:50:04.282130 20538 raft_consensus.cc:348] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:04.282243 20538 raft_consensus.cc:729] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.282308 7274 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:04.282402 20538 consensus_queue.cc:260] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:04.282717 20538 ts_tablet_manager.cc:1398] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240426 09:50:04.283859 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-722306427b7b46b4b0942193c821d990 received from {username='slave'} at 127.0.99.131:56047
I20240426 09:50:04.283932 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-722306427b7b46b4b0942193c821d990 on tablet 722306427b7b46b4b0942193c821d990 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:04.285279 6735 catalog_manager.cc:5484] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:04.344000 20593 ts_tablet_manager.cc:902] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:04.344617 20593 tablet_copy_client.cc:308] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:04.345058 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet b967d83e93d843398e93077285865b1b from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:55679)
I20240426 09:50:04.345173 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet b967d83e93d843398e93077285865b1b from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:55679: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-b967d83e93d843398e93077285865b1b
I20240426 09:50:04.346233 6902 tablet_copy_source_session.cc:205] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.347172 20593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b967d83e93d843398e93077285865b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.349315 20593 tablet_copy_client.cc:783] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.349455 20593 tablet_copy_client.cc:647] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.350490 20593 tablet_copy_client.cc:523] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.352252 20593 tablet_bootstrap.cc:492] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:50:04.358450 20593 tablet_bootstrap.cc:492] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.358704 20593 tablet_bootstrap.cc:492] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:50:04.358883 20593 ts_tablet_manager.cc:1367] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240426 09:50:04.359246 20593 raft_consensus.cc:348] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:04.359342 20593 raft_consensus.cc:729] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.359454 20593 consensus_queue.cc:260] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:04.359714 20593 ts_tablet_manager.cc:1398] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:04.360217 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-b967d83e93d843398e93077285865b1b received from {username='slave'} at 127.0.99.135:55679
I20240426 09:50:04.360296 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-b967d83e93d843398e93077285865b1b on tablet b967d83e93d843398e93077285865b1b with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:50:04.431350 20611 ts_tablet_manager.cc:902] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:04.432005 20611 tablet_copy_client.cc:308] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:04.432801 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 1d720846739e49f888da92befe06a309 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:32985)
I20240426 09:50:04.432912 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 1d720846739e49f888da92befe06a309 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:32985: session id = 558c4ecd0ece460d9591aa7a3c799d19-1d720846739e49f888da92befe06a309
I20240426 09:50:04.434000 7164 tablet_copy_source_session.cc:205] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.435765 20611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d720846739e49f888da92befe06a309. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.440196 20611 tablet_copy_client.cc:783] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.440380 20611 tablet_copy_client.cc:647] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.442716 20611 tablet_copy_client.cc:523] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.444228 20611 tablet_bootstrap.cc:492] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:50:04.446702 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:04.447185 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:04.450073 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:04.450580 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:04.451653 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:04.452994 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:04.461864 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:04.472003 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:04.485034 20611 tablet_bootstrap.cc:492] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.485245 20611 tablet_bootstrap.cc:492] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:50:04.486807 20611 ts_tablet_manager.cc:1367] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.043s user 0.005s sys 0.000s
I20240426 09:50:04.490222 20611 raft_consensus.cc:348] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:04.490494 20611 raft_consensus.cc:729] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.490644 20611 consensus_queue.cc:260] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:04.491685 20611 ts_tablet_manager.cc:1398] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:50:04.492152 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-1d720846739e49f888da92befe06a309 received from {username='slave'} at 127.0.99.136:32985
I20240426 09:50:04.492238 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-1d720846739e49f888da92befe06a309 on tablet 1d720846739e49f888da92befe06a309 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:50:04.495585 20617 ts_tablet_manager.cc:902] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:04.496979 20618 ts_tablet_manager.cc:902] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:04.497505 20618 tablet_copy_client.cc:308] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:04.502876 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 948e1df8d52f42cebcd289fff377d6fb from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:40097)
I20240426 09:50:04.502987 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 948e1df8d52f42cebcd289fff377d6fb from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:40097: session id = 3b41d87e7bfb419f8aa7500100129b6c-948e1df8d52f42cebcd289fff377d6fb
I20240426 09:50:04.503762 6902 tablet_copy_source_session.cc:205] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.504633 20618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 948e1df8d52f42cebcd289fff377d6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.507035 20618 tablet_copy_client.cc:783] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.507174 20618 tablet_copy_client.cc:647] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.507520 20617 tablet_copy_client.cc:308] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:04.508524 20618 tablet_copy_client.cc:523] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.509871 20618 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:04.512111 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet 6dce6f1e165c492ab23022527bac580a from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:59303)
I20240426 09:50:04.512204 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet 6dce6f1e165c492ab23022527bac580a from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:59303: session id = c244e02f3cf847c0a5b69e1059f1071c-6dce6f1e165c492ab23022527bac580a
I20240426 09:50:04.513074 9437 tablet_copy_source_session.cc:205] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.513932 20617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dce6f1e165c492ab23022527bac580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.516243 20617 tablet_copy_client.cc:783] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.516376 20617 tablet_copy_client.cc:647] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.518342 20617 tablet_copy_client.cc:523] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.519865 20538 ts_tablet_manager.cc:902] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:04.520284 20538 tablet_copy_client.cc:308] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:04.520787 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet ca0a2b121aba4659bf2d3a475a1e298b from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:48359)
I20240426 09:50:04.520872 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet ca0a2b121aba4659bf2d3a475a1e298b from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:48359: session id = 1aaf8befba984e199e96f53f46b50441-ca0a2b121aba4659bf2d3a475a1e298b
I20240426 09:50:04.521824 9685 tablet_copy_source_session.cc:205] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.522838 20538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0a2b121aba4659bf2d3a475a1e298b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.525077 20538 tablet_copy_client.cc:783] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.525161 20617 tablet_bootstrap.cc:492] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:50:04.525234 20538 tablet_copy_client.cc:647] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.529152 20538 tablet_copy_client.cc:523] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.530453 20538 tablet_bootstrap.cc:492] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:04.534083 20538 tablet_bootstrap.cc:492] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.534327 20538 tablet_bootstrap.cc:492] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:04.534489 20538 ts_tablet_manager.cc:1367] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.001s
I20240426 09:50:04.534945 20538 raft_consensus.cc:348] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:04.535079 20538 raft_consensus.cc:729] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.535205 20538 consensus_queue.cc:260] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:04.535497 20538 ts_tablet_manager.cc:1398] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:04.536197 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-ca0a2b121aba4659bf2d3a475a1e298b received from {username='slave'} at 127.0.99.131:48359
I20240426 09:50:04.536285 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-ca0a2b121aba4659bf2d3a475a1e298b on tablet ca0a2b121aba4659bf2d3a475a1e298b with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:04.538050 20617 tablet_bootstrap.cc:492] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.538297 20617 tablet_bootstrap.cc:492] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:50:04.538460 20617 ts_tablet_manager.cc:1367] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20240426 09:50:04.538975 20617 raft_consensus.cc:348] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:04.539099 20617 raft_consensus.cc:729] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.539244 20617 consensus_queue.cc:260] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:04.539624 20617 ts_tablet_manager.cc:1398] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:50:04.541996 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-6dce6f1e165c492ab23022527bac580a received from {username='slave'} at 127.0.99.129:59303
I20240426 09:50:04.542078 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-6dce6f1e165c492ab23022527bac580a on tablet 6dce6f1e165c492ab23022527bac580a with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:50:04.545231 20618 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.545521 20618 tablet_bootstrap.cc:492] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:04.545688 20618 ts_tablet_manager.cc:1367] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.036s user 0.022s sys 0.004s
I20240426 09:50:04.546527 20618 raft_consensus.cc:348] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:04.546630 20618 raft_consensus.cc:729] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.546809 20618 consensus_queue.cc:260] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:04.547082 20618 ts_tablet_manager.cc:1398] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240426 09:50:04.547614 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-948e1df8d52f42cebcd289fff377d6fb received from {username='slave'} at 127.0.99.134:40097
I20240426 09:50:04.547699 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-948e1df8d52f42cebcd289fff377d6fb on tablet 948e1df8d52f42cebcd289fff377d6fb with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:04.559770 20645 ts_tablet_manager.cc:902] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:04.560442 20645 tablet_copy_client.cc:308] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:04.560966 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 838bef0a35c046e39b465f2ea743350f from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:37061)
I20240426 09:50:04.561075 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 838bef0a35c046e39b465f2ea743350f from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:37061: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-838bef0a35c046e39b465f2ea743350f
I20240426 09:50:04.562132 7164 tablet_copy_source_session.cc:205] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.563006 20645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838bef0a35c046e39b465f2ea743350f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.565347 20645 tablet_copy_client.cc:783] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.565501 20645 tablet_copy_client.cc:647] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.571229 20645 tablet_copy_client.cc:523] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.573181 20645 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:04.577831 20645 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.578234 20645 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:04.578387 20645 ts_tablet_manager.cc:1367] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240426 09:50:04.578855 20645 raft_consensus.cc:348] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:04.578969 20645 raft_consensus.cc:729] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.579157 20645 consensus_queue.cc:260] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:04.579618 20645 ts_tablet_manager.cc:1398] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:04.580057 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-838bef0a35c046e39b465f2ea743350f received from {username='slave'} at 127.0.99.130:37061
I20240426 09:50:04.580132 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-838bef0a35c046e39b465f2ea743350f on tablet 838bef0a35c046e39b465f2ea743350f with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:04.648437 20645 ts_tablet_manager.cc:902] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:04.648965 20645 tablet_copy_client.cc:308] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:04.649621 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet d2a5223682934aa698dd23aa6eab941b from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:58979)
I20240426 09:50:04.649746 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet d2a5223682934aa698dd23aa6eab941b from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:58979: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-d2a5223682934aa698dd23aa6eab941b
I20240426 09:50:04.650960 9437 tablet_copy_source_session.cc:205] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.651857 20645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2a5223682934aa698dd23aa6eab941b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.653940 20645 tablet_copy_client.cc:783] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.654089 20645 tablet_copy_client.cc:647] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.657450 20645 tablet_copy_client.cc:523] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.658666 20645 tablet_bootstrap.cc:492] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:04.662202 20645 tablet_bootstrap.cc:492] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.662448 20645 tablet_bootstrap.cc:492] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:04.662606 20645 ts_tablet_manager.cc:1367] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:50:04.663080 20645 raft_consensus.cc:348] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:04.663205 20645 raft_consensus.cc:729] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.663376 20645 consensus_queue.cc:260] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:04.663683 20645 ts_tablet_manager.cc:1398] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:50:04.664518 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-d2a5223682934aa698dd23aa6eab941b received from {username='slave'} at 127.0.99.130:58979
I20240426 09:50:04.664597 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-d2a5223682934aa698dd23aa6eab941b on tablet d2a5223682934aa698dd23aa6eab941b with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:04.703091 20618 ts_tablet_manager.cc:902] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:04.703550 20618 tablet_copy_client.cc:308] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:04.703979 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet e6cb4534d05c49ed859b4abe61097ac8 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:43705)
I20240426 09:50:04.704083 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet e6cb4534d05c49ed859b4abe61097ac8 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:43705: session id = 3b41d87e7bfb419f8aa7500100129b6c-e6cb4534d05c49ed859b4abe61097ac8
I20240426 09:50:04.705021 9306 tablet_copy_source_session.cc:205] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.705919 20618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6cb4534d05c49ed859b4abe61097ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.708323 20618 tablet_copy_client.cc:783] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.708487 20618 tablet_copy_client.cc:647] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.708693 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:04.709584 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:04.711323 20618 tablet_copy_client.cc:523] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.713989 20611 ts_tablet_manager.cc:902] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:04.714433 20611 tablet_copy_client.cc:308] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:04.716023 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 5ba744a156204d9e81db6ea113945203 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:32985)
I20240426 09:50:04.716123 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 5ba744a156204d9e81db6ea113945203 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:32985: session id = 558c4ecd0ece460d9591aa7a3c799d19-5ba744a156204d9e81db6ea113945203
I20240426 09:50:04.717032 7164 tablet_copy_source_session.cc:205] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.718663 20618 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:04.720434 20611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ba744a156204d9e81db6ea113945203. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.722913 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:04.723492 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:04.726914 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:04.727713 9546 raft_consensus.cc:1203] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:04.740798 20618 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.741927 20618 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:04.742190 20618 ts_tablet_manager.cc:1367] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.000s
I20240426 09:50:04.742905 20618 raft_consensus.cc:348] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:04.743036 20618 raft_consensus.cc:729] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.743184 20618 consensus_queue.cc:260] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:04.743836 20618 ts_tablet_manager.cc:1398] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:50:04.745661 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-e6cb4534d05c49ed859b4abe61097ac8 received from {username='slave'} at 127.0.99.134:43705
I20240426 09:50:04.745755 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-e6cb4534d05c49ed859b4abe61097ac8 on tablet e6cb4534d05c49ed859b4abe61097ac8 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:04.747237 20707 ts_tablet_manager.cc:902] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:04.747879 20707 tablet_copy_client.cc:308] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:04.748414 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 9c94c820ab65420a8d3f885ccc9f8884 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:44679)
I20240426 09:50:04.748523 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 9c94c820ab65420a8d3f885ccc9f8884 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:44679: session id = 558c4ecd0ece460d9591aa7a3c799d19-9c94c820ab65420a8d3f885ccc9f8884
I20240426 09:50:04.749011 20611 tablet_copy_client.cc:783] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.749164 20611 tablet_copy_client.cc:647] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.749728 7033 tablet_copy_source_session.cc:205] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.752220 20611 tablet_copy_client.cc:523] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.753733 20611 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:50:04.755895 20707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c94c820ab65420a8d3f885ccc9f8884. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.760866 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:04.762081 20707 tablet_copy_client.cc:783] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.762259 20707 tablet_copy_client.cc:647] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.764194 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:04.780937 20617 ts_tablet_manager.cc:902] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:04.781363 20617 tablet_copy_client.cc:308] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:50:04.784329 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet 11604d6436a946eba63a36c28e1b0290 from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:35867)
I20240426 09:50:04.784446 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet 11604d6436a946eba63a36c28e1b0290 from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:35867: session id = c244e02f3cf847c0a5b69e1059f1071c-11604d6436a946eba63a36c28e1b0290
I20240426 09:50:04.785274 9569 tablet_copy_source_session.cc:205] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.787890 20617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11604d6436a946eba63a36c28e1b0290. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.790210 20617 tablet_copy_client.cc:783] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.790380 20617 tablet_copy_client.cc:647] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.793555 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:04.806603 20611 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.806932 20611 tablet_bootstrap.cc:492] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:50:04.807205 20611 ts_tablet_manager.cc:1367] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.054s user 0.014s sys 0.014s
I20240426 09:50:04.807696 20611 raft_consensus.cc:348] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:04.807819 20611 raft_consensus.cc:729] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.807957 20611 consensus_queue.cc:260] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:04.809123 20611 ts_tablet_manager.cc:1398] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:50:04.809561 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-5ba744a156204d9e81db6ea113945203 received from {username='slave'} at 127.0.99.136:32985
I20240426 09:50:04.809655 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-5ba744a156204d9e81db6ea113945203 on tablet 5ba744a156204d9e81db6ea113945203 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:50:04.810300 20617 tablet_copy_client.cc:523] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.810739 7143 raft_consensus.cc:1203] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:04.811931 20617 tablet_bootstrap.cc:492] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:50:04.818724 20617 tablet_bootstrap.cc:492] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.819029 20617 tablet_bootstrap.cc:492] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:50:04.819224 20617 ts_tablet_manager.cc:1367] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20240426 09:50:04.819747 20617 raft_consensus.cc:348] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:04.819869 20617 raft_consensus.cc:729] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.819983 20617 consensus_queue.cc:260] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:04.820258 20617 ts_tablet_manager.cc:1398] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:04.820958 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-11604d6436a946eba63a36c28e1b0290 received from {username='slave'} at 127.0.99.129:35867
I20240426 09:50:04.821043 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-11604d6436a946eba63a36c28e1b0290 on tablet 11604d6436a946eba63a36c28e1b0290 with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:50:04.825824 20707 tablet_copy_client.cc:523] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.827350 20707 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:50:04.854707 20707 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.855057 20707 tablet_bootstrap.cc:492] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:50:04.855228 20707 ts_tablet_manager.cc:1367] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.012s
I20240426 09:50:04.855672 20707 raft_consensus.cc:348] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:04.855798 20707 raft_consensus.cc:729] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.855908 20707 consensus_queue.cc:260] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:04.856190 20707 ts_tablet_manager.cc:1398] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:04.857254 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-9c94c820ab65420a8d3f885ccc9f8884 received from {username='slave'} at 127.0.99.136:44679
I20240426 09:50:04.857336 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-9c94c820ab65420a8d3f885ccc9f8884 on tablet 9c94c820ab65420a8d3f885ccc9f8884 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:50:04.867010 20618 ts_tablet_manager.cc:902] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:04.867449 20618 tablet_copy_client.cc:308] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:04.867902 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 5e010e7f54b54b04afe4e3b1df0c739f from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:52457)
I20240426 09:50:04.868021 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 5e010e7f54b54b04afe4e3b1df0c739f from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:52457: session id = 3b41d87e7bfb419f8aa7500100129b6c-5e010e7f54b54b04afe4e3b1df0c739f
I20240426 09:50:04.869145 7164 tablet_copy_source_session.cc:205] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:04.869920 20618 ts_tablet_manager.cc:1875] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240426 09:50:04.872558 20618 ts_tablet_manager.cc:1888] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20240426 09:50:04.872747 20618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e010e7f54b54b04afe4e3b1df0c739f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:04.873802 20618 tablet_copy_client.cc:783] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:04.873962 20618 tablet_copy_client.cc:647] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:04.875164 20618 tablet_copy_client.cc:523] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:04.877022 20618 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:04.892264 20618 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:04.892551 20618 tablet_bootstrap.cc:492] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:04.892730 20618 ts_tablet_manager.cc:1367] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.001s
I20240426 09:50:04.893267 20618 raft_consensus.cc:348] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:04.893393 20618 raft_consensus.cc:729] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:04.893577 20618 consensus_queue.cc:260] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:04.895668 20618 ts_tablet_manager.cc:1398] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20240426 09:50:04.899312 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-5e010e7f54b54b04afe4e3b1df0c739f received from {username='slave'} at 127.0.99.134:52457
I20240426 09:50:04.899407 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-5e010e7f54b54b04afe4e3b1df0c739f on tablet 5e010e7f54b54b04afe4e3b1df0c739f with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:04.940322 9417 raft_consensus.cc:1203] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240426 09:50:04.960085 9672 raft_consensus.cc:1203] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:04.990139 7013 raft_consensus.cc:1203] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:04.995440 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:05.002326 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:05.002575 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:05.004227 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:05.008409 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:05.023851 6881 raft_consensus.cc:1203] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:05.030383 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:05.039314 7142 raft_consensus.cc:1203] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2] Dedup: 2.2->[]
I20240426 09:50:05.041785 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:05.053344 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:05.228345 7013 raft_consensus.cc:1203] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:05.229177 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:05.231635 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:05.231650 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:05.240397 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:05.242367 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:05.247069 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:05.256871 9669 raft_consensus.cc:1203] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240426 09:50:05.258646 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:05.268674 6880 raft_consensus.cc:1203] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:05.272739 9416 raft_consensus.cc:1203] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:05.288246 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:05.338166 9669 raft_consensus.cc:1203] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240426 09:50:05.338583 9416 consensus_queue.cc:237] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:05.339792 7013 raft_consensus.cc:1261] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:05.340396 20232 consensus_queue.cc:1035] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:05.341094 6881 raft_consensus.cc:1261] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:05.341464 20413 consensus_queue.cc:1035] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:05.342543 20232 raft_consensus.cc:2932] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:05.342974 7013 raft_consensus.cc:2932] T d2a5223682934aa698dd23aa6eab941b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:05.345474 6881 raft_consensus.cc:2932] T d2a5223682934aa698dd23aa6eab941b P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:05.346601 6739 catalog_manager.cc:5484] T d2a5223682934aa698dd23aa6eab941b P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:05.352105 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet d2a5223682934aa698dd23aa6eab941b with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:05.352770 20851 tablet_replica.cc:330] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:05.352929 20851 raft_consensus.cc:2229] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:05.353061 20851 raft_consensus.cc:2258] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:05.353596 20851 ts_tablet_manager.cc:1875] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:05.356155 20851 ts_tablet_manager.cc:1888] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:05.356225 20851 log.cc:1196] T d2a5223682934aa698dd23aa6eab941b P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/d2a5223682934aa698dd23aa6eab941b
I20240426 09:50:05.356746 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet d2a5223682934aa698dd23aa6eab941b (table rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]) successfully deleted
I20240426 09:50:05.395124 9416 raft_consensus.cc:1203] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:05.482465 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:05.484833 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:05.486349 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:05.495222 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:05.508028 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:05.513121 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:05.531376 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:05.538908 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:05.611393 9546 consensus_queue.cc:237] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:05.612665 7013 raft_consensus.cc:1261] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:05.612690 6881 raft_consensus.cc:1261] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:05.613034 20489 consensus_queue.cc:1035] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:05.613178 19919 consensus_queue.cc:1035] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:05.614823 19919 raft_consensus.cc:2932] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:05.615314 7013 raft_consensus.cc:2932] T 11604d6436a946eba63a36c28e1b0290 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:05.619246 6733 catalog_manager.cc:5484] T 11604d6436a946eba63a36c28e1b0290 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:05.620580 6881 raft_consensus.cc:2932] T 11604d6436a946eba63a36c28e1b0290 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:05.625473 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 11604d6436a946eba63a36c28e1b0290 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:05.630980 20921 tablet_replica.cc:330] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:05.631207 20921 raft_consensus.cc:2229] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:05.631386 20921 raft_consensus.cc:2258] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:05.631982 20921 ts_tablet_manager.cc:1875] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:05.641561 20921 ts_tablet_manager.cc:1888] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:05.641669 20921 log.cc:1196] T 11604d6436a946eba63a36c28e1b0290 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/11604d6436a946eba63a36c28e1b0290
I20240426 09:50:05.642891 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 11604d6436a946eba63a36c28e1b0290 (table rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]) successfully deleted
I20240426 09:50:05.768589 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:05.779075 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:05.780272 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:05.782234 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:05.787117 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:05.787353 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:05.799620 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:05.822068 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:05.891041 7013 consensus_queue.cc:237] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:05.892220 9416 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:05.892278 9669 raft_consensus.cc:1261] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:05.892760 20454 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:05.892946 20681 consensus_queue.cc:1035] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:05.894793 20681 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:05.895267 9416 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:05.897823 9669 raft_consensus.cc:2932] T 9c94c820ab65420a8d3f885ccc9f8884 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:05.899609 6735 catalog_manager.cc:5484] T 9c94c820ab65420a8d3f885ccc9f8884 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 6 to 7, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:05.902685 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c94c820ab65420a8d3f885ccc9f8884 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:05.903085 20990 tablet_replica.cc:330] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:05.903412 20990 raft_consensus.cc:2229] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:05.903671 20990 raft_consensus.cc:2258] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:05.904358 20990 ts_tablet_manager.cc:1875] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:05.907096 20990 ts_tablet_manager.cc:1888] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240426 09:50:05.907229 20990 log.cc:1196] T 9c94c820ab65420a8d3f885ccc9f8884 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/9c94c820ab65420a8d3f885ccc9f8884
I20240426 09:50:05.907927 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 9c94c820ab65420a8d3f885ccc9f8884 (table rebalance_test_table_3 [id=dae0f9a006424f1f8b3beb6d17e2eefa]) successfully deleted
I20240426 09:50:06.060544 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:06.061371 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:06.065752 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:06.066047 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:06.066895 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:06.066972 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:06.080552 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:06.084026 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:06.192111 7143 consensus_queue.cc:237] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:06.193513 7013 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:06.193521 9546 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:06.193931 19205 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:06.194123 18470 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:06.196107 21063 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:06.196436 9546 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:06.196532 7013 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:06.200143 6738 catalog_manager.cc:5484] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:06.203783 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 838bef0a35c046e39b465f2ea743350f with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:06.204982 21069 tablet_replica.cc:330] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:06.205150 21069 raft_consensus.cc:2229] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:06.205279 21069 raft_consensus.cc:2258] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:06.205693 21069 ts_tablet_manager.cc:1875] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:06.208451 21069 ts_tablet_manager.cc:1888] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:06.208544 21069 log.cc:1196] T 838bef0a35c046e39b465f2ea743350f P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/838bef0a35c046e39b465f2ea743350f
I20240426 09:50:06.209228 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 838bef0a35c046e39b465f2ea743350f (table rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]) successfully deleted
I20240426 09:50:06.332532 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:06.333271 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:06.337817 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:06.343468 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:06.346807 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:06.349326 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:06.375792 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:06.398622 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:06.445225 9417 consensus_queue.cc:237] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:06.446520 6881 raft_consensus.cc:1261] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:06.446696 9672 raft_consensus.cc:1261] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:06.446898 21043 consensus_queue.cc:1035] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:06.447093 20232 consensus_queue.cc:1035] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:06.448767 20413 raft_consensus.cc:2932] T 6dce6f1e165c492ab23022527bac580a P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:06.449322 6881 raft_consensus.cc:2932] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:06.449309 9672 raft_consensus.cc:2932] T 6dce6f1e165c492ab23022527bac580a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:06.453200 6735 catalog_manager.cc:5484] T 6dce6f1e165c492ab23022527bac580a P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:06.456315 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dce6f1e165c492ab23022527bac580a with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:06.456666 21138 tablet_replica.cc:330] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:06.456822 21138 raft_consensus.cc:2229] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:06.456954 21138 raft_consensus.cc:2258] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:06.457396 21138 ts_tablet_manager.cc:1875] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:06.460304 21138 ts_tablet_manager.cc:1888] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:06.460438 21138 log.cc:1196] T 6dce6f1e165c492ab23022527bac580a P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/6dce6f1e165c492ab23022527bac580a
I20240426 09:50:06.461717 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 6dce6f1e165c492ab23022527bac580a (table rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]) successfully deleted
I20240426 09:50:06.599002 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:06.603971 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:06.610085 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:06.611176 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:06.615180 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:06.621974 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:06.645921 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:06.658663 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:06.732563 9672 consensus_queue.cc:237] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:06.733917 7143 raft_consensus.cc:1261] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:06.733928 7274 raft_consensus.cc:1261] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:06.734427 18652 consensus_queue.cc:1035] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:06.734675 21137 consensus_queue.cc:1035] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:06.736454 21209 raft_consensus.cc:2932] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:06.736922 7274 raft_consensus.cc:2932] T ca0a2b121aba4659bf2d3a475a1e298b P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:06.737020 7143 raft_consensus.cc:2932] T ca0a2b121aba4659bf2d3a475a1e298b P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:06.741151 6737 catalog_manager.cc:5484] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:06.744318 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0a2b121aba4659bf2d3a475a1e298b with delete_type TABLET_DATA_TOMBSTONED (TS c244e02f3cf847c0a5b69e1059f1071c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44196
I20240426 09:50:06.745108 21214 tablet_replica.cc:330] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:50:06.745376 21214 raft_consensus.cc:2229] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:06.745536 21214 raft_consensus.cc:2258] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:06.746042 21214 ts_tablet_manager.cc:1875] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:06.748510 21214 ts_tablet_manager.cc:1888] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.2
I20240426 09:50:06.748623 21214 log.cc:1196] T ca0a2b121aba4659bf2d3a475a1e298b P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/ca0a2b121aba4659bf2d3a475a1e298b
I20240426 09:50:06.749383 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet ca0a2b121aba4659bf2d3a475a1e298b (table rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]) successfully deleted
I20240426 09:50:06.873032 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:06.878713 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:06.879204 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:06.893721 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:06.896001 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:06.904322 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:06.905385 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:06.920163 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:06.997625 6880 consensus_queue.cc:237] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:06.998943 9546 raft_consensus.cc:1261] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:06.998943 7275 raft_consensus.cc:1261] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:06.999310 20337 consensus_queue.cc:1035] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:06.999533 21117 consensus_queue.cc:1035] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:07.001422 21117 raft_consensus.cc:2932] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:07.001762 7275 raft_consensus.cc:2932] T b967d83e93d843398e93077285865b1b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:07.005070 6733 catalog_manager.cc:5484] T b967d83e93d843398e93077285865b1b P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:07.007730 9546 raft_consensus.cc:2932] T b967d83e93d843398e93077285865b1b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:07.008531 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet b967d83e93d843398e93077285865b1b with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47654
I20240426 09:50:07.008926 21285 tablet_replica.cc:330] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:50:07.009105 21285 raft_consensus.cc:2229] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:07.009328 21285 raft_consensus.cc:2258] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:07.010211 21285 ts_tablet_manager.cc:1875] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:07.013870 21285 ts_tablet_manager.cc:1888] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:07.013976 21285 log.cc:1196] T b967d83e93d843398e93077285865b1b P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/b967d83e93d843398e93077285865b1b
I20240426 09:50:07.014619 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet b967d83e93d843398e93077285865b1b (table rebalancer_extra_table_44 [id=c95b130ebf7d47d4b859ca3b4e6ed495]) successfully deleted
I20240426 09:50:07.144834 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:07.146286 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:07.146471 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:07.147296 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:07.172129 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:07.173497 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:07.177127 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:07.197455 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:07.261013 6880 consensus_queue.cc:237] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:07.262442 9416 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:07.262540 9286 raft_consensus.cc:1261] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:07.262864 20337 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:07.263216 20337 consensus_queue.cc:1035] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:07.264905 20337 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:07.265724 9286 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:07.266372 9416 raft_consensus.cc:2932] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:07.270313 6740 catalog_manager.cc:5484] T 948e1df8d52f42cebcd289fff377d6fb P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:07.274227 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 948e1df8d52f42cebcd289fff377d6fb with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:07.274601 21358 tablet_replica.cc:330] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:07.274752 21358 raft_consensus.cc:2229] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:07.274992 21358 raft_consensus.cc:2258] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:07.275537 21358 ts_tablet_manager.cc:1875] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:07.278183 21358 ts_tablet_manager.cc:1888] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240426 09:50:07.278263 21358 log.cc:1196] T 948e1df8d52f42cebcd289fff377d6fb P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/948e1df8d52f42cebcd289fff377d6fb
I20240426 09:50:07.278944 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 948e1df8d52f42cebcd289fff377d6fb (table rebalance_test_table_11 [id=6d560c3b62a748c292935b9a5f0c5d66]) successfully deleted
I20240426 09:50:07.403347 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:07.414172 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:07.423174 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:07.423861 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:07.427826 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:07.439242 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:07.447306 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:07.469311 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:07.516963 7274 consensus_queue.cc:237] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:07.518344 7142 raft_consensus.cc:1261] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:07.518642 6880 raft_consensus.cc:1261] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:07.518805 21422 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:07.519174 21422 consensus_queue.cc:1035] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:07.520659 21422 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:07.522956 7142 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:07.524503 6740 catalog_manager.cc:5484] T 722306427b7b46b4b0942193c821d990 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:07.528988 6880 raft_consensus.cc:2932] T 722306427b7b46b4b0942193c821d990 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:07.534161 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 722306427b7b46b4b0942193c821d990 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:07.534499 21430 tablet_replica.cc:330] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:07.534737 21430 raft_consensus.cc:2229] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:07.535043 21430 raft_consensus.cc:2258] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:07.535471 21430 ts_tablet_manager.cc:1875] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:07.537858 21430 ts_tablet_manager.cc:1888] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:07.537921 21430 log.cc:1196] T 722306427b7b46b4b0942193c821d990 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/722306427b7b46b4b0942193c821d990
I20240426 09:50:07.538386 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 722306427b7b46b4b0942193c821d990 (table rebalancer_extra_table_2 [id=338527113d944bce981b1e3edafc6c83]) successfully deleted
I20240426 09:50:07.732749 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:07.736205 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:07.754858 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:07.758051 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:07.760949 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:07.770941 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:07.822305 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:07.831468 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:07.993645 9286 consensus_queue.cc:237] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:07.995240 9416 raft_consensus.cc:1261] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:07.995240 9672 raft_consensus.cc:1261] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:07.995687 21059 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:07.995851 21483 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:07.997673 21483 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:07.998044 9416 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:08.000124 9672 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:08.003240 6731 catalog_manager.cc:5484] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:08.007480 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet e6cb4534d05c49ed859b4abe61097ac8 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:08.007728 21430 tablet_replica.cc:330] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:08.007882 21430 raft_consensus.cc:2229] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:08.008040 21430 raft_consensus.cc:2258] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:08.008499 21430 ts_tablet_manager.cc:1875] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:08.013303 21430 ts_tablet_manager.cc:1888] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:08.015906 21430 log.cc:1196] T e6cb4534d05c49ed859b4abe61097ac8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/e6cb4534d05c49ed859b4abe61097ac8
I20240426 09:50:08.016589 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet e6cb4534d05c49ed859b4abe61097ac8 (table rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]) successfully deleted
I20240426 09:50:08.199013 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:08.199710 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:08.199795 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:08.200140 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:08.200425 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:08.209710 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:08.212977 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:08.231709 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:08.304682 7142 consensus_queue.cc:237] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:08.306154 9672 raft_consensus.cc:1261] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:08.306550 6880 raft_consensus.cc:1261] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:08.307004 18470 consensus_queue.cc:1035] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:08.307199 21495 consensus_queue.cc:1035] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:08.309052 21495 raft_consensus.cc:2932] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:08.309459 6880 raft_consensus.cc:2932] T 1d720846739e49f888da92befe06a309 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:08.310910 9669 raft_consensus.cc:2932] T 1d720846739e49f888da92befe06a309 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:08.312531 6734 catalog_manager.cc:5484] T 1d720846739e49f888da92befe06a309 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:08.315563 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d720846739e49f888da92befe06a309 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:08.316813 21588 tablet_replica.cc:330] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:08.316974 21588 raft_consensus.cc:2229] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:08.317222 21588 raft_consensus.cc:2258] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:08.317721 21588 ts_tablet_manager.cc:1875] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:08.320369 21588 ts_tablet_manager.cc:1888] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:08.320461 21588 log.cc:1196] T 1d720846739e49f888da92befe06a309 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/1d720846739e49f888da92befe06a309
I20240426 09:50:08.321116 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 1d720846739e49f888da92befe06a309 (table rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]) successfully deleted
I20240426 09:50:08.348157 9549 consensus_queue.cc:237] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:08.349696 7011 raft_consensus.cc:1261] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:08.349869 9286 raft_consensus.cc:1261] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:08.350438 21615 consensus_queue.cc:1035] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:08.350687 9484 consensus_peers.cc:485] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 753f199b5cb2426ba90e8ad003efcb32. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:08.350941 21616 consensus_queue.cc:1035] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:08.352566 21616 raft_consensus.cc:2932] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:08.352977 7011 raft_consensus.cc:2932] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:08.354470 9286 raft_consensus.cc:2932] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:08.355988 6740 catalog_manager.cc:5484] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:08.380476 9549 consensus_queue.cc:237] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:08.382104 9416 raft_consensus.cc:1261] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:08.382339 9482 consensus_peers.cc:485] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d5f900639e2494b97e13f3703536f42. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:08.382547 9286 raft_consensus.cc:1261] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:08.382733 21615 consensus_queue.cc:1035] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:08.383113 21615 consensus_queue.cc:1035] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:08.386016 21616 raft_consensus.cc:2932] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:08.387158 9286 raft_consensus.cc:2932] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:08.389446 9416 raft_consensus.cc:2932] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:08.390280 6734 catalog_manager.cc:5484] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:08.423244 9286 consensus_queue.cc:237] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:08.424820 9669 raft_consensus.cc:1261] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:08.424882 9416 raft_consensus.cc:1261] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:08.425134 9219 consensus_peers.cc:485] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da33173a2b0c45ba95cebdf61c6a7499. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:08.425335 21517 consensus_queue.cc:1035] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:08.425526 21059 consensus_queue.cc:1035] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:08.427405 21059 raft_consensus.cc:2932] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:08.427845 9416 raft_consensus.cc:2932] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:08.429876 9669 raft_consensus.cc:2932] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:08.432654 6740 catalog_manager.cc:5484] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:08.453804 7142 consensus_queue.cc:237] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:08.455403 7275 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240426 09:50:08.455426 7077 consensus_peers.cc:485] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd377032e7cd4b4998424f8685ad5b85. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:08.455483 9669 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:08.455816 18470 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:08.456046 21495 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:08.457924 21495 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:08.458459 7275 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:08.461536 9669 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:08.461835 6737 catalog_manager.cc:5484] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 5 to 6, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:08.494446 9546 consensus_queue.cc:237] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:08.496182 9416 raft_consensus.cc:1261] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:08.496183 9286 raft_consensus.cc:1261] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:08.496713 20923 consensus_queue.cc:1035] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:08.497318 20923 consensus_queue.cc:1035] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:08.497495 9484 consensus_peers.cc:485] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8cb2e6c9a044463aa58878e06fc3fff2. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:08.499176 21615 raft_consensus.cc:2932] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:08.501266 9416 raft_consensus.cc:2932] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:08.501317 9286 raft_consensus.cc:2932] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:08.504508 6734 catalog_manager.cc:5484] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:08.696740 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:08.702106 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:08.711781 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:08.718282 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:08.720896 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:08.723034 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:08.728468 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:08.758630 21746 ts_tablet_manager.cc:902] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:08.759222 21746 tablet_copy_client.cc:308] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:50:08.759646 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet 753f199b5cb2426ba90e8ad003efcb32 from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:35867)
I20240426 09:50:08.759739 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet 753f199b5cb2426ba90e8ad003efcb32 from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:35867: session id = c244e02f3cf847c0a5b69e1059f1071c-753f199b5cb2426ba90e8ad003efcb32
I20240426 09:50:08.760700 9569 tablet_copy_source_session.cc:205] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:08.764046 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:08.764583 21746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753f199b5cb2426ba90e8ad003efcb32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:08.770646 21746 tablet_copy_client.cc:783] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:08.770854 21746 tablet_copy_client.cc:647] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:08.781601 21746 tablet_copy_client.cc:523] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:08.783203 21746 tablet_bootstrap.cc:492] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:50:08.788183 21746 tablet_bootstrap.cc:492] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:08.788448 21746 tablet_bootstrap.cc:492] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:50:08.788614 21746 ts_tablet_manager.cc:1367] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20240426 09:50:08.789086 21746 raft_consensus.cc:348] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:08.789232 21746 raft_consensus.cc:729] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:50:08.789357 21746 consensus_queue.cc:260] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:08.789659 21746 ts_tablet_manager.cc:1398] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:08.790521 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-753f199b5cb2426ba90e8ad003efcb32 received from {username='slave'} at 127.0.99.129:35867
I20240426 09:50:08.790618 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-753f199b5cb2426ba90e8ad003efcb32 on tablet 753f199b5cb2426ba90e8ad003efcb32 with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:50:08.928486 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:08.930517 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:08.948491 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:08.950855 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:08.959172 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:08.959532 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:08.962160 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:08.975028 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:08.990314 21805 ts_tablet_manager.cc:902] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:08.990890 21805 tablet_copy_client.cc:308] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:08.991168 21806 ts_tablet_manager.cc:902] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:08.991978 21806 tablet_copy_client.cc:308] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:50:08.994838 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet cd377032e7cd4b4998424f8685ad5b85 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:49245)
I20240426 09:50:08.994949 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet cd377032e7cd4b4998424f8685ad5b85 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:49245: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-cd377032e7cd4b4998424f8685ad5b85
I20240426 09:50:08.995904 7164 tablet_copy_source_session.cc:205] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:08.997037 21805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd377032e7cd4b4998424f8685ad5b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:08.998829 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet 3d5f900639e2494b97e13f3703536f42 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:33967)
I20240426 09:50:08.998939 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet 3d5f900639e2494b97e13f3703536f42 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:33967: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-3d5f900639e2494b97e13f3703536f42
I20240426 09:50:09.001399 9569 tablet_copy_source_session.cc:205] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:09.004699 21806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d5f900639e2494b97e13f3703536f42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:09.010286 21805 tablet_copy_client.cc:783] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:09.010504 21805 tablet_copy_client.cc:647] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:09.011798 21805 tablet_copy_client.cc:523] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:09.012249 21806 tablet_copy_client.cc:783] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:09.012403 21806 tablet_copy_client.cc:647] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:09.013690 21805 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:50:09.016396 21806 tablet_copy_client.cc:523] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:09.017752 21806 tablet_bootstrap.cc:492] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:09.020210 21815 ts_tablet_manager.cc:902] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:09.022404 21746 ts_tablet_manager.cc:902] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:09.022908 21746 tablet_copy_client.cc:308] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:50:09.023355 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet 8cb2e6c9a044463aa58878e06fc3fff2 from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:35867)
I20240426 09:50:09.023474 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet 8cb2e6c9a044463aa58878e06fc3fff2 from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:35867: session id = c244e02f3cf847c0a5b69e1059f1071c-8cb2e6c9a044463aa58878e06fc3fff2
I20240426 09:50:09.024520 9569 tablet_copy_source_session.cc:205] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:09.025493 21746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cb2e6c9a044463aa58878e06fc3fff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:09.027509 21746 tablet_copy_client.cc:783] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:09.027658 21746 tablet_copy_client.cc:647] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:09.028378 21806 tablet_bootstrap.cc:492] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:09.028669 21806 tablet_bootstrap.cc:492] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:09.028882 21806 ts_tablet_manager.cc:1367] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20240426 09:50:09.029363 21746 tablet_copy_client.cc:523] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:09.029392 21806 raft_consensus.cc:348] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:09.029523 21806 raft_consensus.cc:729] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:09.029673 21806 consensus_queue.cc:260] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:09.029999 21806 ts_tablet_manager.cc:1398] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:09.030164 21815 tablet_copy_client.cc:308] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:09.030527 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-3d5f900639e2494b97e13f3703536f42 received from {username='slave'} at 127.0.99.130:33967
I20240426 09:50:09.030638 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-3d5f900639e2494b97e13f3703536f42 on tablet 3d5f900639e2494b97e13f3703536f42 with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:09.030712 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet da33173a2b0c45ba95cebdf61c6a7499 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:36823)
I20240426 09:50:09.030869 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet da33173a2b0c45ba95cebdf61c6a7499 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:36823: session id = 1aaf8befba984e199e96f53f46b50441-da33173a2b0c45ba95cebdf61c6a7499
I20240426 09:50:09.031636 21746 tablet_bootstrap.cc:492] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:50:09.031807 9306 tablet_copy_source_session.cc:205] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:09.038079 21815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da33173a2b0c45ba95cebdf61c6a7499. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:09.040763 21815 tablet_copy_client.cc:783] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:09.040989 21815 tablet_copy_client.cc:647] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:09.042572 21815 tablet_copy_client.cc:523] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:09.044108 21815 tablet_bootstrap.cc:492] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:09.046270 21746 tablet_bootstrap.cc:492] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:09.046572 21746 tablet_bootstrap.cc:492] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:50:09.046751 21746 ts_tablet_manager.cc:1367] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20240426 09:50:09.047323 21746 raft_consensus.cc:348] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:09.047451 21746 raft_consensus.cc:729] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:50:09.047583 21746 consensus_queue.cc:260] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:09.047953 21746 ts_tablet_manager.cc:1398] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:09.049011 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-8cb2e6c9a044463aa58878e06fc3fff2 received from {username='slave'} at 127.0.99.129:35867
I20240426 09:50:09.049108 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-8cb2e6c9a044463aa58878e06fc3fff2 on tablet 8cb2e6c9a044463aa58878e06fc3fff2 with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:50:09.049957 21815 tablet_bootstrap.cc:492] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:09.050244 21815 tablet_bootstrap.cc:492] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:09.050407 21815 ts_tablet_manager.cc:1367] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240426 09:50:09.050873 21815 raft_consensus.cc:348] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:09.050988 21815 raft_consensus.cc:729] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:09.051577 21815 consensus_queue.cc:260] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:09.051882 21815 ts_tablet_manager.cc:1398] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:09.052559 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-da33173a2b0c45ba95cebdf61c6a7499 received from {username='slave'} at 127.0.99.131:36823
I20240426 09:50:09.052639 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-da33173a2b0c45ba95cebdf61c6a7499 on tablet da33173a2b0c45ba95cebdf61c6a7499 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:09.064044 21805 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:09.064577 21805 tablet_bootstrap.cc:492] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:50:09.064788 21805 ts_tablet_manager.cc:1367] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.051s user 0.026s sys 0.001s
I20240426 09:50:09.065219 21805 raft_consensus.cc:348] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:09.065330 21805 raft_consensus.cc:729] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:50:09.065443 21805 consensus_queue.cc:260] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:09.065747 21805 ts_tablet_manager.cc:1398] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:09.066293 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-cd377032e7cd4b4998424f8685ad5b85 received from {username='slave'} at 127.0.99.135:49245
I20240426 09:50:09.066382 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-cd377032e7cd4b4998424f8685ad5b85 on tablet cd377032e7cd4b4998424f8685ad5b85 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:50:09.200095 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:09.213742 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:09.217777 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:09.224181 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:09.226758 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:09.233598 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:09.244259 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:09.246158 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:09.337714 6881 raft_consensus.cc:1203] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:09.377857 7011 raft_consensus.cc:1203] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:09.429821 9546 raft_consensus.cc:1203] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240426 09:50:09.452050 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:09.464895 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:09.468185 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:09.477303 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:09.478237 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:09.480387 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:09.498275 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:09.519052 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:09.570066 6881 raft_consensus.cc:1203] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:09.585675 7142 raft_consensus.cc:1203] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:09.593115 7142 consensus_queue.cc:237] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:09.594439 9672 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:09.594439 9546 raft_consensus.cc:1261] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:09.594899 21675 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:09.595019 18470 consensus_queue.cc:1035] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:09.597010 18470 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:09.597375 9546 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:09.600883 6738 catalog_manager.cc:5484] T cd377032e7cd4b4998424f8685ad5b85 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:09.602699 9672 raft_consensus.cc:2932] T cd377032e7cd4b4998424f8685ad5b85 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:09.604295 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet cd377032e7cd4b4998424f8685ad5b85 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:09.606570 21956 tablet_replica.cc:330] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:09.606745 21956 raft_consensus.cc:2229] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:09.606954 21956 raft_consensus.cc:2258] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:09.607422 21956 ts_tablet_manager.cc:1875] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:09.610430 21956 ts_tablet_manager.cc:1888] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240426 09:50:09.610513 21956 log.cc:1196] T cd377032e7cd4b4998424f8685ad5b85 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/cd377032e7cd4b4998424f8685ad5b85
I20240426 09:50:09.611052 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet cd377032e7cd4b4998424f8685ad5b85 (table rebalance_test_table_19 [id=4b7d3f1e96474ec8884dfcaa1088d3aa]) successfully deleted
I20240426 09:50:09.739235 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:09.758718 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:09.760828 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:09.763327 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:09.766803 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:09.768252 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:09.768604 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:09.787663 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:09.860414 7142 consensus_queue.cc:237] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:09.861868 9416 raft_consensus.cc:1261] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:09.862089 7275 raft_consensus.cc:1261] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:09.862387 21675 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:09.862583 18470 consensus_queue.cc:1035] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:09.864573 21675 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:09.865108 9416 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:09.865334 7275 raft_consensus.cc:2932] T 5e010e7f54b54b04afe4e3b1df0c739f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:09.868816 6736 catalog_manager.cc:5484] T 5e010e7f54b54b04afe4e3b1df0c739f P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 4 to 5, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:09.872373 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e010e7f54b54b04afe4e3b1df0c739f with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:09.872937 22029 tablet_replica.cc:330] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:09.873104 22029 raft_consensus.cc:2229] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:09.873696 22029 raft_consensus.cc:2258] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:09.874173 22029 ts_tablet_manager.cc:1875] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:09.877182 22029 ts_tablet_manager.cc:1888] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:09.877338 22029 log.cc:1196] T 5e010e7f54b54b04afe4e3b1df0c739f P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/5e010e7f54b54b04afe4e3b1df0c739f
I20240426 09:50:09.877948 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 5e010e7f54b54b04afe4e3b1df0c739f (table rebalancer_extra_table_0 [id=93c34a9d2f2944f2b972d6dd257f2c01]) successfully deleted
I20240426 09:50:10.026010 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:10.029013 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:10.037830 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:10.043725 7123 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:10.046921 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:10.055357 9660 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:10.065829 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:10.077210 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:10.115234 7142 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5ba744a156204d9e81db6ea113945203"
dest_uuid: "1aaf8befba984e199e96f53f46b50441"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46918
I20240426 09:50:10.115439 7142 raft_consensus.cc:595] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:10.158900 21675 raft_consensus.cc:979] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: : Instructing follower 558c4ecd0ece460d9591aa7a3c799d19 to start an election
I20240426 09:50:10.159020 21675 raft_consensus.cc:1067] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Signalling peer 558c4ecd0ece460d9591aa7a3c799d19 to start an election
I20240426 09:50:10.159569 9669 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5ba744a156204d9e81db6ea113945203"
dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
from {username='slave'} at 127.0.99.131:48359
I20240426 09:50:10.159729 9669 raft_consensus.cc:482] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:10.159826 9669 raft_consensus.cc:3037] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:10.160955 9669 raft_consensus.cc:504] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:10.161442 9669 leader_election.cc:288] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 2 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:10.161863 7142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba744a156204d9e81db6ea113945203" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:50:10.161986 7142 raft_consensus.cc:3032] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:10.162029 7142 raft_consensus.cc:729] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:50:10.162206 7142 consensus_queue.cc:260] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:10.162377 7142 raft_consensus.cc:3037] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:10.162402 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba744a156204d9e81db6ea113945203" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:50:10.162521 9286 raft_consensus.cc:3037] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:10.163271 7142 raft_consensus.cc:2454] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 2.
I20240426 09:50:10.163523 9286 raft_consensus.cc:2454] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 2.
I20240426 09:50:10.163858 9615 leader_election.cc:302] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1aaf8befba984e199e96f53f46b50441, 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:50:10.164031 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ba744a156204d9e81db6ea113945203" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:50:10.164163 7275 raft_consensus.cc:3037] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:10.165148 7275 raft_consensus.cc:2454] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 2.
I20240426 09:50:10.165355 21826 raft_consensus.cc:2781] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:10.165452 21826 raft_consensus.cc:686] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:50:10.165596 21826 consensus_queue.cc:237] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:10.168525 6735 catalog_manager.cc:5484] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 1 to 2, leader changed from 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 2 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:10.258944 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:10.264380 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:10.272193 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:10.275447 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:10.286711 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:10.286952 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:10.289451 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:10.291370 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:10.475504 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:10.484675 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:10.484946 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:10.490159 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:10.491809 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:10.521050 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:10.524884 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:10.525154 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:10.560726 7142 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:10.561168 21826 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:10.603739 9549 consensus_queue.cc:237] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:10.605062 6881 raft_consensus.cc:1261] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:10.605458 7013 raft_consensus.cc:1261] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:10.605652 21616 consensus_queue.cc:1035] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:10.606252 21616 consensus_queue.cc:1035] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:10.608054 20923 raft_consensus.cc:2932] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:10.608651 6881 raft_consensus.cc:2932] T 753f199b5cb2426ba90e8ad003efcb32 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:10.608817 7013 raft_consensus.cc:2932] T 753f199b5cb2426ba90e8ad003efcb32 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:10.611794 6739 catalog_manager.cc:5484] T 753f199b5cb2426ba90e8ad003efcb32 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:10.612596 9286 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:10.614955 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 753f199b5cb2426ba90e8ad003efcb32 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:10.615093 22190 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:10.619074 7275 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:10.619457 21497 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:10.622399 22209 tablet_replica.cc:330] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:10.622614 22209 raft_consensus.cc:2229] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:10.622802 22209 raft_consensus.cc:2258] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:10.623627 22209 ts_tablet_manager.cc:1875] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:10.635397 22209 ts_tablet_manager.cc:1888] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:10.635499 22209 log.cc:1196] T 753f199b5cb2426ba90e8ad003efcb32 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/753f199b5cb2426ba90e8ad003efcb32
I20240426 09:50:10.636332 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 753f199b5cb2426ba90e8ad003efcb32 (table rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]) successfully deleted
I20240426 09:50:10.722738 6881 consensus_queue.cc:237] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:10.724898 7142 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:10.725009 9549 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20240426 09:50:10.725477 6815 consensus_peers.cc:485] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53765d0ffcc24bb98aadc0cd0fe44af3. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:10.725692 21933 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:10.725925 22287 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:10.728116 22287 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:10.728545 9549 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:10.733224 6732 catalog_manager.cc:5484] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 2 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:10.735599 7142 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:10.759757 7142 consensus_queue.cc:237] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:10.761247 9549 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240426 09:50:10.761282 7077 consensus_peers.cc:485] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0362b7d68d9048bca47d96e948563d03. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:10.761287 7275 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:10.761695 22306 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:10.761866 18470 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:10.763626 22306 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:10.764063 9549 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:10.765898 7275 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:10.768810 6732 catalog_manager.cc:5484] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 5 to 6, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:10.790298 6881 consensus_queue.cc:237] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:10.791806 9672 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:10.791774 7275 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240426 09:50:10.791929 6816 consensus_peers.cc:485] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5fe43b0765d9401a88c8e9a028ab3ef4. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:10.792168 21508 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:10.792348 22287 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:10.794431 21508 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:10.794921 7275 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:10.796806 9672 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:10.798991 6740 catalog_manager.cc:5484] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:10.822853 9417 consensus_queue.cc:237] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:10.825259 9286 raft_consensus.cc:1261] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:10.825392 9672 raft_consensus.cc:1261] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:10.825793 22345 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:10.826012 9350 consensus_peers.cc:485] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 957533c35bae4299b35b77ba9a465eb3. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:10.826015 21804 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:10.828020 21804 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:10.828470 9672 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:10.832510 9286 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:10.832818 6739 catalog_manager.cc:5484] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:10.856628 9417 consensus_queue.cc:237] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:10.858361 9286 raft_consensus.cc:1261] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:10.858508 9549 raft_consensus.cc:1261] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:10.858757 20232 consensus_queue.cc:1035] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:10.858968 9350 consensus_peers.cc:485] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c623acd7cc9480faec733d2fd109117. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:10.858968 22345 consensus_queue.cc:1035] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:10.861385 22345 raft_consensus.cc:2932] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:10.863404 9286 raft_consensus.cc:2932] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:10.865574 6739 catalog_manager.cc:5484] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:10.866549 9549 raft_consensus.cc:2932] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:10.895620 9286 consensus_queue.cc:237] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:10.897622 7275 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:10.897653 9219 consensus_peers.cc:485] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: efa472b0759c4cbbb7db13dc3fbe0b50. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:10.898018 21998 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:10.898207 7011 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:10.899722 21998 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:10.901562 22383 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:10.901959 7011 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:10.903961 7275 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:10.907181 6732 catalog_manager.cc:5484] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:10.929051 9546 consensus_queue.cc:237] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:10.930735 7142 raft_consensus.cc:1261] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:10.930977 6881 raft_consensus.cc:1261] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:10.931149 22403 consensus_queue.cc:1035] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:10.931392 20923 consensus_queue.cc:1035] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:10.931581 9483 consensus_peers.cc:485] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f1bb1cdfa9684023ad017c0e36cb44b1. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:10.933512 20923 raft_consensus.cc:2932] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:10.934051 6881 raft_consensus.cc:2932] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:10.935914 7142 raft_consensus.cc:2932] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:10.939846 6740 catalog_manager.cc:5484] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:10.979821 7011 consensus_queue.cc:237] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:10.981412 9417 raft_consensus.cc:1261] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:10.981714 6946 consensus_peers.cc:485] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 10118b6d060f4999a2cdfdaa1b8de197. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:10.981997 22030 consensus_queue.cc:1035] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:10.982164 9672 raft_consensus.cc:1261] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:10.982532 22030 consensus_queue.cc:1035] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:10.984212 20454 raft_consensus.cc:2932] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:10.985916 9672 raft_consensus.cc:2932] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:10.986032 9417 raft_consensus.cc:2932] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:10.991109 6739 catalog_manager.cc:5484] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:11.013303 9672 consensus_queue.cc:237] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:11.014997 9417 raft_consensus.cc:1261] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:11.015177 7142 raft_consensus.cc:1261] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:11.015211 9616 consensus_peers.cc:485] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c10b7673a7bf4980add3d94c6284e57b. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:11.015559 21826 consensus_queue.cc:1035] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:11.015848 22190 consensus_queue.cc:1035] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:11.018127 21826 raft_consensus.cc:2932] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:11.019981 7142 raft_consensus.cc:2932] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:11.020042 9417 raft_consensus.cc:2932] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:11.021731 6732 catalog_manager.cc:5484] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:11.043901 9672 consensus_queue.cc:237] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:11.045370 7142 raft_consensus.cc:1261] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:11.045656 9615 consensus_peers.cc:485] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360aa06557974c5ea635a32245c0903e. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:11.046144 9546 raft_consensus.cc:1261] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:11.046236 22190 consensus_queue.cc:1035] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:11.046806 21826 consensus_queue.cc:1035] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:11.048593 21826 raft_consensus.cc:2932] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:11.048960 7142 raft_consensus.cc:2932] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:11.053237 6740 catalog_manager.cc:5484] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:11.059248 9546 raft_consensus.cc:2932] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:11.091766 9286 consensus_queue.cc:237] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:11.093248 7275 raft_consensus.cc:1261] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:11.093397 9221 consensus_peers.cc:485] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87d356a7ac32469c824af0970730214f. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:11.093639 21998 consensus_queue.cc:1035] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:11.093925 9672 raft_consensus.cc:1261] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:11.094516 21998 consensus_queue.cc:1035] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:11.096729 21517 raft_consensus.cc:2932] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:11.097162 9672 raft_consensus.cc:2932] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:11.100297 7275 raft_consensus.cc:2932] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:11.101073 6739 catalog_manager.cc:5484] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:11.159929 22523 ts_tablet_manager.cc:902] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:11.160989 22523 tablet_copy_client.cc:308] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:11.161510 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 0362b7d68d9048bca47d96e948563d03 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:52457)
I20240426 09:50:11.161654 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 0362b7d68d9048bca47d96e948563d03 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:52457: session id = 3b41d87e7bfb419f8aa7500100129b6c-0362b7d68d9048bca47d96e948563d03
I20240426 09:50:11.163022 7164 tablet_copy_source_session.cc:205] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.163941 22523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0362b7d68d9048bca47d96e948563d03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.166118 22523 tablet_copy_client.cc:783] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.166270 22523 tablet_copy_client.cc:647] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.167652 22523 tablet_copy_client.cc:523] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.169014 22523 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:11.215093 22523 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.215447 22523 tablet_bootstrap.cc:492] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:11.215667 22523 ts_tablet_manager.cc:1367] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.047s user 0.028s sys 0.000s
I20240426 09:50:11.216143 22523 raft_consensus.cc:348] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:11.216279 22523 raft_consensus.cc:729] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.216432 22523 consensus_queue.cc:260] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:11.216814 22523 ts_tablet_manager.cc:1398] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:50:11.218693 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-0362b7d68d9048bca47d96e948563d03 received from {username='slave'} at 127.0.99.134:52457
I20240426 09:50:11.218814 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-0362b7d68d9048bca47d96e948563d03 on tablet 0362b7d68d9048bca47d96e948563d03 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:11.244537 22529 ts_tablet_manager.cc:902] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:11.246311 22529 tablet_copy_client.cc:308] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:11.246829 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 5fe43b0765d9401a88c8e9a028ab3ef4 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:54839)
I20240426 09:50:11.246953 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 5fe43b0765d9401a88c8e9a028ab3ef4 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:54839: session id = a692602e1a2e4794877e6fb92a7b9962-5fe43b0765d9401a88c8e9a028ab3ef4
I20240426 09:50:11.247983 6902 tablet_copy_source_session.cc:205] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.248936 22529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fe43b0765d9401a88c8e9a028ab3ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.252303 22529 tablet_copy_client.cc:783] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.252461 22529 tablet_copy_client.cc:647] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.257422 22529 tablet_copy_client.cc:523] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.260769 22529 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:11.262828 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:11.275059 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:11.276103 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:11.283776 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:11.285646 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:11.291713 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:11.302199 22547 ts_tablet_manager.cc:902] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:11.302955 22547 tablet_copy_client.cc:308] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:11.303618 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet 8c623acd7cc9480faec733d2fd109117 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:58979)
I20240426 09:50:11.303737 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet 8c623acd7cc9480faec733d2fd109117 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:58979: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-8c623acd7cc9480faec733d2fd109117
I20240426 09:50:11.304821 9437 tablet_copy_source_session.cc:205] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.305723 22547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c623acd7cc9480faec733d2fd109117. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.309585 22547 tablet_copy_client.cc:783] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.309789 22547 tablet_copy_client.cc:647] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.314968 22548 ts_tablet_manager.cc:902] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:11.316013 22529 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.316347 22529 tablet_bootstrap.cc:492] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:11.316531 22529 ts_tablet_manager.cc:1367] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.056s user 0.026s sys 0.000s
I20240426 09:50:11.316987 22529 raft_consensus.cc:348] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:11.317145 22529 raft_consensus.cc:729] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.317270 22529 consensus_queue.cc:260] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:11.317560 22529 ts_tablet_manager.cc:1398] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:11.320679 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-5fe43b0765d9401a88c8e9a028ab3ef4 received from {username='slave'} at 127.0.99.133:54839
I20240426 09:50:11.320760 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-5fe43b0765d9401a88c8e9a028ab3ef4 on tablet 5fe43b0765d9401a88c8e9a028ab3ef4 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:11.320986 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:11.321944 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:11.325692 22547 tablet_copy_client.cc:523] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.327453 22547 tablet_bootstrap.cc:492] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:11.328532 22548 tablet_copy_client.cc:308] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:11.330335 22529 ts_tablet_manager.cc:902] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:11.330716 22529 tablet_copy_client.cc:308] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:50:11.331584 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet f1bb1cdfa9684023ad017c0e36cb44b1 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:57057)
I20240426 09:50:11.331691 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet f1bb1cdfa9684023ad017c0e36cb44b1 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:57057: session id = a692602e1a2e4794877e6fb92a7b9962-f1bb1cdfa9684023ad017c0e36cb44b1
I20240426 09:50:11.332672 9569 tablet_copy_source_session.cc:205] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.333405 22547 tablet_bootstrap.cc:492] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.333652 22547 tablet_bootstrap.cc:492] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:11.333814 22547 ts_tablet_manager.cc:1367] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20240426 09:50:11.334347 22547 raft_consensus.cc:348] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:11.334467 22547 raft_consensus.cc:729] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.334605 22547 consensus_queue.cc:260] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:11.334965 22547 ts_tablet_manager.cc:1398] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.001s sys 0.002s
I20240426 09:50:11.335613 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-8c623acd7cc9480faec733d2fd109117 received from {username='slave'} at 127.0.99.130:58979
I20240426 09:50:11.335711 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-8c623acd7cc9480faec733d2fd109117 on tablet 8c623acd7cc9480faec733d2fd109117 with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:11.336181 22529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1bb1cdfa9684023ad017c0e36cb44b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.338027 22529 tablet_copy_client.cc:783] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.338172 22529 tablet_copy_client.cc:647] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.340229 22529 tablet_copy_client.cc:523] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.341830 22529 tablet_bootstrap.cc:492] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:11.342595 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 53765d0ffcc24bb98aadc0cd0fe44af3 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:34321)
I20240426 09:50:11.342698 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 53765d0ffcc24bb98aadc0cd0fe44af3 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:34321: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-53765d0ffcc24bb98aadc0cd0fe44af3
I20240426 09:50:11.343744 6902 tablet_copy_source_session.cc:205] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.350169 22529 tablet_bootstrap.cc:492] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.350430 22529 tablet_bootstrap.cc:492] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:11.350592 22529 ts_tablet_manager.cc:1367] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20240426 09:50:11.351068 22529 raft_consensus.cc:348] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:11.351193 22529 raft_consensus.cc:729] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.351259 22548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53765d0ffcc24bb98aadc0cd0fe44af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.351310 22529 consensus_queue.cc:260] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:11.351559 22529 ts_tablet_manager.cc:1398] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:50:11.352700 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-f1bb1cdfa9684023ad017c0e36cb44b1 received from {username='slave'} at 127.0.99.133:57057
I20240426 09:50:11.352809 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-f1bb1cdfa9684023ad017c0e36cb44b1 on tablet f1bb1cdfa9684023ad017c0e36cb44b1 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:11.356140 22548 tablet_copy_client.cc:783] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.356381 22548 tablet_copy_client.cc:647] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.357470 22548 tablet_copy_client.cc:523] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.358762 22548 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:11.358862 22564 ts_tablet_manager.cc:902] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:11.359362 22564 tablet_copy_client.cc:308] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:11.359874 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet efa472b0759c4cbbb7db13dc3fbe0b50 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:59663)
I20240426 09:50:11.359983 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet efa472b0759c4cbbb7db13dc3fbe0b50 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:59663: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-efa472b0759c4cbbb7db13dc3fbe0b50
I20240426 09:50:11.360886 9306 tablet_copy_source_session.cc:205] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.362735 22564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa472b0759c4cbbb7db13dc3fbe0b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.367719 22564 tablet_copy_client.cc:783] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.367906 22564 tablet_copy_client.cc:647] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.369114 22564 tablet_copy_client.cc:523] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.370455 22564 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:50:11.378476 22564 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.379390 22564 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:50:11.379576 22564 ts_tablet_manager.cc:1367] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.001s
I20240426 09:50:11.380048 22564 raft_consensus.cc:348] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:11.380177 22564 raft_consensus.cc:729] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.380369 22564 consensus_queue.cc:260] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:11.380726 22564 ts_tablet_manager.cc:1398] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:11.381577 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-efa472b0759c4cbbb7db13dc3fbe0b50 received from {username='slave'} at 127.0.99.135:59663
I20240426 09:50:11.381664 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-efa472b0759c4cbbb7db13dc3fbe0b50 on tablet efa472b0759c4cbbb7db13dc3fbe0b50 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:50:11.381999 22548 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.382292 22548 tablet_bootstrap.cc:492] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:11.382467 22548 ts_tablet_manager.cc:1367] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.003s
I20240426 09:50:11.382946 22548 raft_consensus.cc:348] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:11.383061 22548 raft_consensus.cc:729] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.383213 22548 consensus_queue.cc:260] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:11.385599 22548 ts_tablet_manager.cc:1398] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:50:11.386137 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-53765d0ffcc24bb98aadc0cd0fe44af3 received from {username='slave'} at 127.0.99.130:34321
I20240426 09:50:11.386230 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-53765d0ffcc24bb98aadc0cd0fe44af3 on tablet 53765d0ffcc24bb98aadc0cd0fe44af3 with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:11.402961 22577 ts_tablet_manager.cc:902] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:11.403427 22577 tablet_copy_client.cc:308] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:11.403836 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet 957533c35bae4299b35b77ba9a465eb3 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:35921)
I20240426 09:50:11.403975 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet 957533c35bae4299b35b77ba9a465eb3 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:35921: session id = 1aaf8befba984e199e96f53f46b50441-957533c35bae4299b35b77ba9a465eb3
I20240426 09:50:11.404973 9437 tablet_copy_source_session.cc:205] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.405858 22577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957533c35bae4299b35b77ba9a465eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.408264 22577 tablet_copy_client.cc:783] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.408427 22577 tablet_copy_client.cc:647] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.409516 22577 tablet_copy_client.cc:523] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.411056 22577 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:11.414144 22577 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.414382 22577 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:11.414543 22577 ts_tablet_manager.cc:1367] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:50:11.416030 22577 raft_consensus.cc:348] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:11.416152 22577 raft_consensus.cc:729] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.416283 22577 consensus_queue.cc:260] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:11.416646 22577 ts_tablet_manager.cc:1398] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:11.417243 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-957533c35bae4299b35b77ba9a465eb3 received from {username='slave'} at 127.0.99.131:35921
I20240426 09:50:11.417313 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-957533c35bae4299b35b77ba9a465eb3 on tablet 957533c35bae4299b35b77ba9a465eb3 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:11.492916 22577 ts_tablet_manager.cc:902] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:11.493389 22577 tablet_copy_client.cc:308] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:11.493844 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 10118b6d060f4999a2cdfdaa1b8de197 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:54895)
I20240426 09:50:11.493995 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 10118b6d060f4999a2cdfdaa1b8de197 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:54895: session id = 1aaf8befba984e199e96f53f46b50441-10118b6d060f4999a2cdfdaa1b8de197
I20240426 09:50:11.495077 7033 tablet_copy_source_session.cc:205] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.496199 22577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10118b6d060f4999a2cdfdaa1b8de197. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.498641 22577 tablet_copy_client.cc:783] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.500068 22577 tablet_copy_client.cc:647] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.501987 22577 tablet_copy_client.cc:523] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.503583 22577 tablet_bootstrap.cc:492] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:11.508668 22577 tablet_bootstrap.cc:492] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.508934 22577 tablet_bootstrap.cc:492] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:11.509107 22577 ts_tablet_manager.cc:1367] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20240426 09:50:11.510377 22577 raft_consensus.cc:348] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:11.510502 22577 raft_consensus.cc:729] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.510977 22577 consensus_queue.cc:260] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:11.514318 22577 ts_tablet_manager.cc:1398] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20240426 09:50:11.515484 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-10118b6d060f4999a2cdfdaa1b8de197 received from {username='slave'} at 127.0.99.131:54895
I20240426 09:50:11.515571 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-10118b6d060f4999a2cdfdaa1b8de197 on tablet 10118b6d060f4999a2cdfdaa1b8de197 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:11.547225 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:11.558491 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:11.568758 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:11.573033 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:11.575018 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:11.575067 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:11.577010 9417 raft_consensus.cc:1203] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240426 09:50:11.584194 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:11.603684 22635 ts_tablet_manager.cc:902] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:11.604446 22635 tablet_copy_client.cc:308] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:11.608655 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet 87d356a7ac32469c824af0970730214f from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:36267)
I20240426 09:50:11.608762 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet 87d356a7ac32469c824af0970730214f from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:36267: session id = c244e02f3cf847c0a5b69e1059f1071c-87d356a7ac32469c824af0970730214f
I20240426 09:50:11.609645 9306 tablet_copy_source_session.cc:205] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.610754 22635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87d356a7ac32469c824af0970730214f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.624840 22635 tablet_copy_client.cc:783] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.625187 22635 tablet_copy_client.cc:647] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.626405 22635 tablet_copy_client.cc:523] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.628403 22635 tablet_bootstrap.cc:492] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:50:11.632045 22635 tablet_bootstrap.cc:492] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.632287 22635 tablet_bootstrap.cc:492] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:50:11.632465 22635 ts_tablet_manager.cc:1367] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240426 09:50:11.632915 22635 raft_consensus.cc:348] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:11.633020 22635 raft_consensus.cc:729] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.633167 22635 consensus_queue.cc:260] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:11.633453 22635 ts_tablet_manager.cc:1398] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:11.633859 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:11.634507 9284 raft_consensus.cc:1203] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240426 09:50:11.640121 22640 ts_tablet_manager.cc:902] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:11.640437 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-87d356a7ac32469c824af0970730214f received from {username='slave'} at 127.0.99.129:36267
I20240426 09:50:11.640530 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-87d356a7ac32469c824af0970730214f on tablet 87d356a7ac32469c824af0970730214f with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:50:11.640702 22640 tablet_copy_client.cc:308] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:11.641243 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet c10b7673a7bf4980add3d94c6284e57b from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:32841)
I20240426 09:50:11.641355 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet c10b7673a7bf4980add3d94c6284e57b from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:32841: session id = e3394a4be9ea4def8aa39b333c82c5ff-c10b7673a7bf4980add3d94c6284e57b
I20240426 09:50:11.642621 9685 tablet_copy_source_session.cc:205] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.647264 22640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c10b7673a7bf4980add3d94c6284e57b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.650604 22529 ts_tablet_manager.cc:902] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:11.651118 22529 tablet_copy_client.cc:308] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:11.651710 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 360aa06557974c5ea635a32245c0903e from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:33703)
I20240426 09:50:11.651829 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 360aa06557974c5ea635a32245c0903e from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:33703: session id = a692602e1a2e4794877e6fb92a7b9962-360aa06557974c5ea635a32245c0903e
I20240426 09:50:11.652742 9685 tablet_copy_source_session.cc:205] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:11.653584 22529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360aa06557974c5ea635a32245c0903e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:11.655948 22529 tablet_copy_client.cc:783] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.656098 22529 tablet_copy_client.cc:647] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.657519 22529 tablet_copy_client.cc:523] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.659013 22529 tablet_bootstrap.cc:492] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:11.662075 22640 tablet_copy_client.cc:783] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:11.662267 22640 tablet_copy_client.cc:647] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:11.662998 22529 tablet_bootstrap.cc:492] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.663241 22529 tablet_bootstrap.cc:492] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:11.663390 22529 ts_tablet_manager.cc:1367] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240426 09:50:11.663790 22529 raft_consensus.cc:348] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:11.663911 22529 raft_consensus.cc:729] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.664027 22529 consensus_queue.cc:260] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:11.664332 22529 ts_tablet_manager.cc:1398] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:11.666863 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-360aa06557974c5ea635a32245c0903e received from {username='slave'} at 127.0.99.133:33703
I20240426 09:50:11.666985 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-360aa06557974c5ea635a32245c0903e on tablet 360aa06557974c5ea635a32245c0903e with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:11.686839 22640 tablet_copy_client.cc:523] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:11.688185 22640 tablet_bootstrap.cc:492] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:11.692245 22640 tablet_bootstrap.cc:492] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:11.692587 22640 tablet_bootstrap.cc:492] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:11.692852 22640 ts_tablet_manager.cc:1367] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240426 09:50:11.693349 22640 raft_consensus.cc:348] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:11.693558 22640 raft_consensus.cc:729] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:11.693764 22640 consensus_queue.cc:260] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:11.694267 22640 ts_tablet_manager.cc:1398] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:11.696481 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-c10b7673a7bf4980add3d94c6284e57b received from {username='slave'} at 127.0.99.132:32841
I20240426 09:50:11.696575 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-c10b7673a7bf4980add3d94c6284e57b on tablet c10b7673a7bf4980add3d94c6284e57b with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:11.735045 7011 raft_consensus.cc:1203] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.4->[2.5-2.5] Dedup: 2.5->[]
I20240426 09:50:11.775090 9549 raft_consensus.cc:1203] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:11.840885 9284 raft_consensus.cc:1203] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:11.852974 7142 raft_consensus.cc:1203] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:11.863142 7011 raft_consensus.cc:1203] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:11.918067 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:11.924703 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:11.930928 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:11.939018 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:11.942653 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:11.951149 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:11.959067 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:11.975482 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:12.012450 6881 raft_consensus.cc:1203] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:12.074255 7142 raft_consensus.cc:1203] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:12.149894 9284 raft_consensus.cc:1203] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:12.155728 7274 raft_consensus.cc:1203] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:12.220235 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:12.223922 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:12.226231 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:12.235874 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:12.237212 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:12.249222 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:12.251070 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:12.265254 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:12.319880 9546 consensus_queue.cc:237] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:12.321178 7143 raft_consensus.cc:1261] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:12.321746 9286 raft_consensus.cc:1261] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:12.322151 21616 consensus_queue.cc:1035] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:12.323086 21616 consensus_queue.cc:1035] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:12.324334 21616 raft_consensus.cc:2932] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:12.326015 9286 raft_consensus.cc:2932] T f1bb1cdfa9684023ad017c0e36cb44b1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:12.326117 7143 raft_consensus.cc:2932] T f1bb1cdfa9684023ad017c0e36cb44b1 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:12.329667 6733 catalog_manager.cc:5484] T f1bb1cdfa9684023ad017c0e36cb44b1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:12.333235 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet f1bb1cdfa9684023ad017c0e36cb44b1 with delete_type TABLET_DATA_TOMBSTONED (TS c244e02f3cf847c0a5b69e1059f1071c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44196
I20240426 09:50:12.334192 22785 tablet_replica.cc:330] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:50:12.334388 22785 raft_consensus.cc:2229] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:12.334702 22785 raft_consensus.cc:2258] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:12.335261 22785 ts_tablet_manager.cc:1875] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:12.339337 22785 ts_tablet_manager.cc:1888] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:12.339481 22785 log.cc:1196] T f1bb1cdfa9684023ad017c0e36cb44b1 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/f1bb1cdfa9684023ad017c0e36cb44b1
I20240426 09:50:12.340459 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet f1bb1cdfa9684023ad017c0e36cb44b1 (table rebalancer_extra_table_30 [id=a94189c52da54ba2bda79bc81da7349f]) successfully deleted
I20240426 09:50:12.472730 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:12.487638 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:12.493798 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:12.497093 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:12.497435 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:12.500278 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:12.515594 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:12.520474 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:12.586706 9417 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8c623acd7cc9480faec733d2fd109117"
dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:42006
I20240426 09:50:12.586941 9417 raft_consensus.cc:595] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:12.710343 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:12.711643 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:12.719128 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:12.727851 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:12.728140 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:12.728508 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:12.749460 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:12.773862 9527 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:12.790383 21804 raft_consensus.cc:979] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: : Instructing follower b8e863bc0edf4940bbf3e9f9b2bbb3e5 to start an election
I20240426 09:50:12.790500 21804 raft_consensus.cc:1067] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Signalling peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 to start an election
I20240426 09:50:12.791046 9546 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8c623acd7cc9480faec733d2fd109117"
dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
from {username='slave'} at 127.0.99.134:39323
I20240426 09:50:12.791179 9546 raft_consensus.cc:482] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:12.791260 9546 raft_consensus.cc:3037] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:12.792150 9546 raft_consensus.cc:504] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:12.792634 9546 leader_election.cc:288] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 2 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:12.793010 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623acd7cc9480faec733d2fd109117" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:50:12.793130 9417 raft_consensus.cc:3032] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:12.793171 9417 raft_consensus.cc:729] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:50:12.793306 9417 consensus_queue.cc:260] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:12.793454 9417 raft_consensus.cc:3037] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:12.794466 9417 raft_consensus.cc:2454] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 2.
I20240426 09:50:12.794884 9286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623acd7cc9480faec733d2fd109117" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:50:12.794999 9286 raft_consensus.cc:3037] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:12.795953 9286 raft_consensus.cc:2454] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 2.
I20240426 09:50:12.796470 9482 leader_election.cc:302] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:50:12.796638 22776 raft_consensus.cc:2781] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:12.796743 22776 raft_consensus.cc:686] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 LEADER]: Becoming Leader. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Running, Role: LEADER
I20240426 09:50:12.796747 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623acd7cc9480faec733d2fd109117" candidate_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:50:12.796878 7011 raft_consensus.cc:3037] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:12.796880 22776 consensus_queue.cc:237] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:12.797776 7011 raft_consensus.cc:2454] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e863bc0edf4940bbf3e9f9b2bbb3e5 in term 2.
I20240426 09:50:12.802453 6735 catalog_manager.cc:5484] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: term changed from 1 to 2, leader changed from 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) to b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135). New cstate: current_term: 2 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:12.940143 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:12.941431 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:12.955859 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:12.959296 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:12.964151 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:12.979431 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:12.991394 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:13.009670 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:13.068066 9416 consensus_queue.cc:237] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:13.069473 9286 raft_consensus.cc:1261] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:13.069507 7142 raft_consensus.cc:1261] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:13.070065 21804 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:13.070272 20232 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:13.072297 21804 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:13.074271 9286 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:13.076794 6731 catalog_manager.cc:5484] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:13.079288 7142 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:13.080057 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 957533c35bae4299b35b77ba9a465eb3 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:13.081862 22958 tablet_replica.cc:330] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:13.082072 22958 raft_consensus.cc:2229] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:13.082299 22958 raft_consensus.cc:2258] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:13.082862 22958 ts_tablet_manager.cc:1875] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:13.085503 22958 ts_tablet_manager.cc:1888] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:13.085647 22958 log.cc:1196] T 957533c35bae4299b35b77ba9a465eb3 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/957533c35bae4299b35b77ba9a465eb3
I20240426 09:50:13.086305 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 957533c35bae4299b35b77ba9a465eb3 (table rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]) successfully deleted
I20240426 09:50:13.226222 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:13.227721 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:13.229224 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:13.232609 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:13.239264 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:13.248904 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:13.251940 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:13.266677 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:13.276592 9286 raft_consensus.cc:1261] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:13.277971 22407 consensus_queue.cc:1035] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:13.326576 9669 consensus_queue.cc:237] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:13.327953 7143 raft_consensus.cc:1261] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:13.328076 7275 raft_consensus.cc:1261] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:13.328529 22190 consensus_queue.cc:1035] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:13.328752 21826 consensus_queue.cc:1035] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:13.330826 22190 raft_consensus.cc:2932] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:13.332106 7013 raft_consensus.cc:1261] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:13.332779 7143 raft_consensus.cc:2932] T c10b7673a7bf4980add3d94c6284e57b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:13.333376 22407 consensus_queue.cc:1035] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:13.337059 6731 catalog_manager.cc:5484] T c10b7673a7bf4980add3d94c6284e57b P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:13.337316 9416 raft_consensus.cc:1261] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:13.337697 23032 consensus_queue.cc:1035] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:13.341377 7275 raft_consensus.cc:2932] T c10b7673a7bf4980add3d94c6284e57b P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:13.343720 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet c10b7673a7bf4980add3d94c6284e57b with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:13.347384 23041 tablet_replica.cc:330] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:13.347587 23041 raft_consensus.cc:2229] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:13.347721 23041 raft_consensus.cc:2258] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:13.348112 23041 ts_tablet_manager.cc:1875] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:13.351709 23041 ts_tablet_manager.cc:1888] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:13.351804 23041 log.cc:1196] T c10b7673a7bf4980add3d94c6284e57b P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/c10b7673a7bf4980add3d94c6284e57b
I20240426 09:50:13.352339 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet c10b7673a7bf4980add3d94c6284e57b (table rebalancer_extra_table_42 [id=6daff3a30786455aaf67a937354a9bbc]) successfully deleted
I20240426 09:50:13.495215 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:13.500108 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:13.504894 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:13.506969 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:13.508252 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:13.521898 6862 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:13.532119 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:13.537240 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:13.596091 6881 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5fe43b0765d9401a88c8e9a028ab3ef4"
dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39074
I20240426 09:50:13.596227 6881 raft_consensus.cc:595] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:13.682320 22693 raft_consensus.cc:979] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: : Instructing follower a692602e1a2e4794877e6fb92a7b9962 to start an election
I20240426 09:50:13.682442 22693 raft_consensus.cc:1067] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Signalling peer a692602e1a2e4794877e6fb92a7b9962 to start an election
I20240426 09:50:13.682948 9286 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5fe43b0765d9401a88c8e9a028ab3ef4"
dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
from {username='slave'} at 127.0.99.129:36267
I20240426 09:50:13.683100 9286 raft_consensus.cc:482] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:13.683154 9286 raft_consensus.cc:3037] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:13.684320 9286 raft_consensus.cc:504] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:13.684794 9286 leader_election.cc:288] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 2 election: Requested vote from peers e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:13.685266 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe43b0765d9401a88c8e9a028ab3ef4" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:50:13.685325 7275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe43b0765d9401a88c8e9a028ab3ef4" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:50:13.685387 6881 raft_consensus.cc:3032] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:13.685437 6881 raft_consensus.cc:729] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:50:13.685458 7275 raft_consensus.cc:3037] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:13.685565 6881 consensus_queue.cc:260] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:13.685755 6881 raft_consensus.cc:3037] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:13.685703 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fe43b0765d9401a88c8e9a028ab3ef4" candidate_uuid: "a692602e1a2e4794877e6fb92a7b9962" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:50:13.685855 9669 raft_consensus.cc:3037] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:13.686270 7275 raft_consensus.cc:2454] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 2.
I20240426 09:50:13.686734 6881 raft_consensus.cc:2454] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 2.
I20240426 09:50:13.686831 9669 raft_consensus.cc:2454] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a692602e1a2e4794877e6fb92a7b9962 in term 2.
I20240426 09:50:13.687175 9219 leader_election.cc:302] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 558c4ecd0ece460d9591aa7a3c799d19, a692602e1a2e4794877e6fb92a7b9962, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:50:13.687372 22834 raft_consensus.cc:2781] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:13.687451 22834 raft_consensus.cc:686] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [term 2 LEADER]: Becoming Leader. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:50:13.687588 22834 consensus_queue.cc:237] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:13.700407 6735 catalog_manager.cc:5484] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: term changed from 1 to 2, leader changed from c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) to a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133). New cstate: current_term: 2 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:13.736243 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:13.736467 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:13.740240 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:13.744308 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:13.746562 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:13.769235 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:13.769675 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:13.771325 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:14.044219 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:14.048475 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:14.062712 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:14.073661 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:14.077244 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:14.080761 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:14.107535 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:14.113152 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:14.190259 7143 consensus_queue.cc:237] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:14.191569 9546 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:14.191612 9417 raft_consensus.cc:1261] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:14.192080 22309 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:14.192304 22557 consensus_queue.cc:1035] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:14.194180 22309 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:14.194960 9417 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:14.196065 9546 raft_consensus.cc:2932] T 0362b7d68d9048bca47d96e948563d03 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:14.198158 6732 catalog_manager.cc:5484] T 0362b7d68d9048bca47d96e948563d03 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 6 to 7, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:14.201965 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 0362b7d68d9048bca47d96e948563d03 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:14.202497 23235 tablet_replica.cc:330] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:14.202661 23235 raft_consensus.cc:2229] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:14.202864 23235 raft_consensus.cc:2258] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:14.203446 23235 ts_tablet_manager.cc:1875] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:14.205835 23235 ts_tablet_manager.cc:1888] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240426 09:50:14.205904 23235 log.cc:1196] T 0362b7d68d9048bca47d96e948563d03 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/0362b7d68d9048bca47d96e948563d03
I20240426 09:50:14.206449 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 0362b7d68d9048bca47d96e948563d03 (table rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]) successfully deleted
I20240426 09:50:14.240898 6880 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:14.241504 22705 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:14.246107 7274 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:14.246477 22705 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:14.256630 9672 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:14.257213 22705 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:14.342314 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:14.350147 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:14.351891 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:14.352380 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:14.364097 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:14.364796 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:14.368435 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:14.373533 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:14.460675 6881 consensus_queue.cc:237] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:14.461902 7011 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:14.461971 7143 raft_consensus.cc:1261] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:14.462347 22693 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:14.462523 22287 consensus_queue.cc:1035] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:14.464246 23313 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 5 to 6, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:14.464619 7011 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 5 to 6, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:14.466029 7143 raft_consensus.cc:2932] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 5 to 6, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:14.468299 6731 catalog_manager.cc:5484] T 53765d0ffcc24bb98aadc0cd0fe44af3 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 5 to 6, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 2 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:14.473224 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 53765d0ffcc24bb98aadc0cd0fe44af3 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:14.475028 23314 tablet_replica.cc:330] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:14.475215 23314 raft_consensus.cc:2229] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:14.475414 23314 raft_consensus.cc:2258] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:14.475885 23314 ts_tablet_manager.cc:1875] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:14.478269 23314 ts_tablet_manager.cc:1888] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:50:14.478344 23314 log.cc:1196] T 53765d0ffcc24bb98aadc0cd0fe44af3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/53765d0ffcc24bb98aadc0cd0fe44af3
I20240426 09:50:14.478883 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 53765d0ffcc24bb98aadc0cd0fe44af3 (table rebalancer_extra_table_8 [id=acf04716751f48498c72f227e5a88d17]) successfully deleted
I20240426 09:50:14.630997 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:14.632045 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:14.634577 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:14.638921 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:14.638981 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:14.648166 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:14.654585 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:14.664227 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:14.734989 9284 consensus_queue.cc:237] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:14.736245 9546 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:14.736277 7011 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:14.736632 22834 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:14.736828 22705 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:14.739095 22834 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:14.741328 7011 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:14.741539 9546 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:14.743901 6733 catalog_manager.cc:5484] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:14.749114 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet efa472b0759c4cbbb7db13dc3fbe0b50 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:14.749478 23391 tablet_replica.cc:330] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:14.749744 23391 raft_consensus.cc:2229] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:14.749984 23391 raft_consensus.cc:2258] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:14.750499 23391 ts_tablet_manager.cc:1875] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:14.752882 23391 ts_tablet_manager.cc:1888] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:14.753023 23391 log.cc:1196] T efa472b0759c4cbbb7db13dc3fbe0b50 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/efa472b0759c4cbbb7db13dc3fbe0b50
I20240426 09:50:14.753563 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet efa472b0759c4cbbb7db13dc3fbe0b50 (table rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]) successfully deleted
I20240426 09:50:14.886432 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:14.890069 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:14.896145 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:14.899209 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:14.908030 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:14.918887 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:14.933286 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:14.940428 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:15.012940 9546 consensus_queue.cc:237] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:15.014515 6881 raft_consensus.cc:1261] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:15.014515 9415 raft_consensus.cc:1261] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:15.015045 23032 consensus_queue.cc:1035] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:15.015249 23215 consensus_queue.cc:1035] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:15.017019 23215 raft_consensus.cc:2932] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:15.017200 6881 raft_consensus.cc:2932] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:15.018954 9415 raft_consensus.cc:2932] T 8cb2e6c9a044463aa58878e06fc3fff2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:15.021487 6739 catalog_manager.cc:5484] T 8cb2e6c9a044463aa58878e06fc3fff2 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:15.025000 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cb2e6c9a044463aa58878e06fc3fff2 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:15.025368 23463 tablet_replica.cc:330] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:15.025579 23463 raft_consensus.cc:2229] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:15.026032 23463 raft_consensus.cc:2258] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:15.026487 23463 ts_tablet_manager.cc:1875] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:15.029037 23463 ts_tablet_manager.cc:1888] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:15.029109 23463 log.cc:1196] T 8cb2e6c9a044463aa58878e06fc3fff2 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/8cb2e6c9a044463aa58878e06fc3fff2
I20240426 09:50:15.029661 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 8cb2e6c9a044463aa58878e06fc3fff2 (table rebalancer_extra_table_10 [id=4b9720829dbc436389907ec1d5f050e7]) successfully deleted
I20240426 09:50:15.172211 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:15.173269 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:15.174548 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:15.178936 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:15.182279 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:15.186614 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:15.190598 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:15.223786 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:15.281132 9286 consensus_queue.cc:237] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:15.282426 9415 raft_consensus.cc:1261] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:15.282455 7143 raft_consensus.cc:1261] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:15.282891 22834 consensus_queue.cc:1035] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:15.283075 22705 consensus_queue.cc:1035] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:15.285096 22705 raft_consensus.cc:2932] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:15.288530 7143 raft_consensus.cc:2932] T da33173a2b0c45ba95cebdf61c6a7499 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:15.288806 9415 raft_consensus.cc:2932] T da33173a2b0c45ba95cebdf61c6a7499 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:15.289283 6735 catalog_manager.cc:5484] T da33173a2b0c45ba95cebdf61c6a7499 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:15.293872 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet da33173a2b0c45ba95cebdf61c6a7499 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:15.294292 23538 tablet_replica.cc:330] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:15.294448 23538 raft_consensus.cc:2229] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:15.294643 23538 raft_consensus.cc:2258] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:15.295331 23538 ts_tablet_manager.cc:1875] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:15.298069 23538 ts_tablet_manager.cc:1888] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:15.298147 23538 log.cc:1196] T da33173a2b0c45ba95cebdf61c6a7499 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/da33173a2b0c45ba95cebdf61c6a7499
I20240426 09:50:15.298838 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet da33173a2b0c45ba95cebdf61c6a7499 (table rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]) successfully deleted
I20240426 09:50:15.442220 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:15.444288 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:15.447059 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:15.451237 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:15.453867 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:15.477893 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:15.487515 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:15.494918 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:15.554226 9548 consensus_queue.cc:237] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:15.555657 9284 raft_consensus.cc:1261] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:15.556041 23032 consensus_queue.cc:1035] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:15.556330 7013 raft_consensus.cc:1261] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:15.556705 23032 consensus_queue.cc:1035] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:15.557633 23372 raft_consensus.cc:2932] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:15.558051 9284 raft_consensus.cc:2932] T 3d5f900639e2494b97e13f3703536f42 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:15.558974 7013 raft_consensus.cc:2932] T 3d5f900639e2494b97e13f3703536f42 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:15.561800 6735 catalog_manager.cc:5484] T 3d5f900639e2494b97e13f3703536f42 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:15.564795 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d5f900639e2494b97e13f3703536f42 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:15.566833 23604 tablet_replica.cc:330] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:15.567651 23604 raft_consensus.cc:2229] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:15.567829 23604 raft_consensus.cc:2258] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:15.568256 23604 ts_tablet_manager.cc:1875] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:15.571220 23604 ts_tablet_manager.cc:1888] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:15.571383 23604 log.cc:1196] T 3d5f900639e2494b97e13f3703536f42 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/3d5f900639e2494b97e13f3703536f42
I20240426 09:50:15.572023 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 3d5f900639e2494b97e13f3703536f42 (table rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]) successfully deleted
I20240426 09:50:15.712658 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:15.717659 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:15.719221 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:15.723627 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:15.726717 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:15.728992 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:15.735606 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:15.759064 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:15.817907 9669 consensus_queue.cc:237] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:15.819694 9284 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240426 09:50:15.820286 7275 raft_consensus.cc:1261] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240426 09:50:15.820739 22190 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240426 09:50:15.820917 21826 consensus_queue.cc:1035] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240426 09:50:15.822715 22190 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:15.823320 9284 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:15.823640 7275 raft_consensus.cc:2932] T 5ba744a156204d9e81db6ea113945203 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:15.827113 6735 catalog_manager.cc:5484] T 5ba744a156204d9e81db6ea113945203 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 6 to 8, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 2 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:15.829764 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ba744a156204d9e81db6ea113945203 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:15.830271 23692 tablet_replica.cc:330] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:15.830510 23692 raft_consensus.cc:2229] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:15.830684 23692 raft_consensus.cc:2258] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:15.831367 23692 ts_tablet_manager.cc:1875] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:15.833827 23692 ts_tablet_manager.cc:1888] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20240426 09:50:15.833946 23692 log.cc:1196] T 5ba744a156204d9e81db6ea113945203 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/5ba744a156204d9e81db6ea113945203
I20240426 09:50:15.834568 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 5ba744a156204d9e81db6ea113945203 (table rebalance_test_table_5 [id=6520694ae01349ecb6e4cee579fa0fc5]) successfully deleted
I20240426 09:50:15.856012 6880 consensus_queue.cc:237] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:15.857713 9415 raft_consensus.cc:1261] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:15.857770 9546 raft_consensus.cc:1261] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:15.857810 6817 consensus_peers.cc:485] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 538173a16ff94e70bc8d25b8f1e7c84e. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:15.858114 23656 consensus_queue.cc:1035] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:15.858366 22693 consensus_queue.cc:1035] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:15.860567 23656 raft_consensus.cc:2932] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:15.862128 9548 raft_consensus.cc:2932] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:15.862495 9415 raft_consensus.cc:2932] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:15.867084 6734 catalog_manager.cc:5484] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:15.908098 7013 consensus_queue.cc:237] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
W20240426 09:50:15.909786 6947 consensus_peers.cc:485] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 643fd2d359a14f8eaa8d8a0d0879ef61. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:15.910115 7142 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:15.910264 9672 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240426 09:50:15.910620 23729 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:15.910848 22427 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:15.912735 23729 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 6 to 7, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:15.913105 9672 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 6 to 7, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:15.917296 6737 catalog_manager.cc:5484] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 6 to 7, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:15.917838 7142 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 6 to 7, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:15.936880 7013 consensus_queue.cc:237] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:15.938122 9415 raft_consensus.cc:1261] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:15.938412 9548 raft_consensus.cc:1261] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:15.938536 23729 consensus_queue.cc:1035] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:15.938884 23729 consensus_queue.cc:1035] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:15.940196 6948 consensus_peers.cc:485] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5f5418f85c2b49ce89f82331689e1b30. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:15.941159 23729 raft_consensus.cc:2932] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:15.941546 9548 raft_consensus.cc:2932] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:15.944607 9415 raft_consensus.cc:2932] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:15.946444 6737 catalog_manager.cc:5484] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:15.977510 7013 consensus_queue.cc:237] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:15.978890 9548 raft_consensus.cc:1261] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:15.978945 6880 raft_consensus.cc:1261] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:15.979036 6948 consensus_peers.cc:485] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a31b0f75c2514916a94205ce0289629e. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:15.979401 22427 consensus_queue.cc:1035] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:15.979590 23186 consensus_queue.cc:1035] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:15.981815 22427 raft_consensus.cc:2932] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:15.982254 6880 raft_consensus.cc:2932] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:15.985350 9548 raft_consensus.cc:2932] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:15.986178 6734 catalog_manager.cc:5484] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:16.145427 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:16.148429 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:16.155853 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:16.155900 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:16.158995 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:16.163556 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:16.182999 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:16.193861 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:16.254554 23835 ts_tablet_manager.cc:902] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:16.255144 23835 tablet_copy_client.cc:308] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:16.255589 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 538173a16ff94e70bc8d25b8f1e7c84e from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:36623)
I20240426 09:50:16.255708 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 538173a16ff94e70bc8d25b8f1e7c84e from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:36623: session id = e3394a4be9ea4def8aa39b333c82c5ff-538173a16ff94e70bc8d25b8f1e7c84e
I20240426 09:50:16.256706 6902 tablet_copy_source_session.cc:205] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:16.258265 23835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538173a16ff94e70bc8d25b8f1e7c84e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:16.260677 23835 tablet_copy_client.cc:783] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:16.260840 23835 tablet_copy_client.cc:647] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:16.261883 23835 tablet_copy_client.cc:523] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:16.262980 23835 tablet_bootstrap.cc:492] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:16.267096 23835 tablet_bootstrap.cc:492] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:16.267306 23835 tablet_bootstrap.cc:492] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:16.267452 23835 ts_tablet_manager.cc:1367] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240426 09:50:16.267833 23835 raft_consensus.cc:348] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:16.267931 23835 raft_consensus.cc:729] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:16.268056 23835 consensus_queue.cc:260] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:16.268435 23835 ts_tablet_manager.cc:1398] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:16.268893 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-538173a16ff94e70bc8d25b8f1e7c84e received from {username='slave'} at 127.0.99.132:36623
I20240426 09:50:16.268972 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-538173a16ff94e70bc8d25b8f1e7c84e on tablet 538173a16ff94e70bc8d25b8f1e7c84e with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:16.365471 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:16.366482 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:16.373878 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:16.377290 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:16.386595 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:16.388332 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:16.390188 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:16.393222 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:16.464527 23835 ts_tablet_manager.cc:902] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:16.465008 23835 tablet_copy_client.cc:308] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:16.465499 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet a31b0f75c2514916a94205ce0289629e from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:32839)
I20240426 09:50:16.465607 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet a31b0f75c2514916a94205ce0289629e from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:32839: session id = e3394a4be9ea4def8aa39b333c82c5ff-a31b0f75c2514916a94205ce0289629e
I20240426 09:50:16.466545 7033 tablet_copy_source_session.cc:205] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:16.467411 23835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a31b0f75c2514916a94205ce0289629e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:16.469425 23835 tablet_copy_client.cc:783] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:16.469580 23835 tablet_copy_client.cc:647] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:16.472077 23835 tablet_copy_client.cc:523] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:16.473450 23835 tablet_bootstrap.cc:492] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:16.477167 23835 tablet_bootstrap.cc:492] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:16.477416 23835 tablet_bootstrap.cc:492] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:16.477583 23835 ts_tablet_manager.cc:1367] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:50:16.478036 23835 raft_consensus.cc:348] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:16.478154 23835 raft_consensus.cc:729] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:16.478279 23835 consensus_queue.cc:260] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:16.478556 23835 ts_tablet_manager.cc:1398] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:50:16.479120 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-a31b0f75c2514916a94205ce0289629e received from {username='slave'} at 127.0.99.132:32839
I20240426 09:50:16.479198 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-a31b0f75c2514916a94205ce0289629e on tablet a31b0f75c2514916a94205ce0289629e with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:16.520197 23930 ts_tablet_manager.cc:902] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:16.520875 23930 tablet_copy_client.cc:308] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:16.521206 23935 ts_tablet_manager.cc:902] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:16.521304 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 5f5418f85c2b49ce89f82331689e1b30 from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:51417)
I20240426 09:50:16.521401 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 5f5418f85c2b49ce89f82331689e1b30 from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:51417: session id = c244e02f3cf847c0a5b69e1059f1071c-5f5418f85c2b49ce89f82331689e1b30
I20240426 09:50:16.521678 23935 tablet_copy_client.cc:308] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:16.522380 7033 tablet_copy_source_session.cc:205] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:16.523274 23930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5418f85c2b49ce89f82331689e1b30. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:16.524134 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 643fd2d359a14f8eaa8d8a0d0879ef61 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:57967)
I20240426 09:50:16.524226 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 643fd2d359a14f8eaa8d8a0d0879ef61 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:57967: session id = a692602e1a2e4794877e6fb92a7b9962-643fd2d359a14f8eaa8d8a0d0879ef61
I20240426 09:50:16.525141 7033 tablet_copy_source_session.cc:205] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:16.525929 23930 tablet_copy_client.cc:783] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:16.525966 23935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 643fd2d359a14f8eaa8d8a0d0879ef61. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:16.526083 23930 tablet_copy_client.cc:647] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:16.527882 23930 tablet_copy_client.cc:523] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:16.528337 23935 tablet_copy_client.cc:783] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:16.528473 23935 tablet_copy_client.cc:647] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:16.530930 23930 tablet_bootstrap.cc:492] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:50:16.535714 23930 tablet_bootstrap.cc:492] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:16.535964 23930 tablet_bootstrap.cc:492] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:50:16.536142 23930 ts_tablet_manager.cc:1367] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:50:16.536589 23930 raft_consensus.cc:348] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:16.539324 23930 raft_consensus.cc:729] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:50:16.539458 23930 consensus_queue.cc:260] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:16.539904 23930 ts_tablet_manager.cc:1398] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:50:16.540335 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-5f5418f85c2b49ce89f82331689e1b30 received from {username='slave'} at 127.0.99.129:51417
I20240426 09:50:16.540406 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-5f5418f85c2b49ce89f82331689e1b30 on tablet 5f5418f85c2b49ce89f82331689e1b30 with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:50:16.542119 23935 tablet_copy_client.cc:523] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:16.543749 23935 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:16.593741 23935 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:16.594108 23935 tablet_bootstrap.cc:492] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:16.594316 23935 ts_tablet_manager.cc:1367] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.051s user 0.027s sys 0.003s
I20240426 09:50:16.594995 23935 raft_consensus.cc:348] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:16.595136 23935 raft_consensus.cc:729] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:16.595278 23935 consensus_queue.cc:260] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:16.607653 23935 ts_tablet_manager.cc:1398] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
I20240426 09:50:16.609031 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-643fd2d359a14f8eaa8d8a0d0879ef61 received from {username='slave'} at 127.0.99.133:57967
I20240426 09:50:16.609117 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-643fd2d359a14f8eaa8d8a0d0879ef61 on tablet 643fd2d359a14f8eaa8d8a0d0879ef61 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:16.630146 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:16.647192 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:16.649128 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:16.654630 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:16.660872 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:16.663115 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:16.671069 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:16.681126 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:16.703176 7274 raft_consensus.cc:1203] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:16.851797 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:16.861002 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:16.862617 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:16.864528 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:16.869177 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:16.873059 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:16.882947 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:16.917660 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:16.972209 9284 raft_consensus.cc:1203] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.6->[2.7-2.7] Dedup: 2.7->[]
I20240426 09:50:16.989913 9284 consensus_queue.cc:237] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:16.991158 7274 raft_consensus.cc:1261] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:16.991156 6880 raft_consensus.cc:1261] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:16.991219 6881 raft_consensus.cc:1203] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:16.991580 24011 consensus_queue.cc:1035] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:16.999033 24011 consensus_queue.cc:1035] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:17.001405 24011 raft_consensus.cc:2932] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:17.004453 6881 raft_consensus.cc:2932] T 87d356a7ac32469c824af0970730214f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:17.004469 7274 raft_consensus.cc:2932] T 87d356a7ac32469c824af0970730214f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:17.004902 6735 catalog_manager.cc:5484] T 87d356a7ac32469c824af0970730214f P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:17.010251 7274 raft_consensus.cc:1203] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:17.017756 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 87d356a7ac32469c824af0970730214f with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:17.018106 24047 tablet_replica.cc:330] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:17.018260 24047 raft_consensus.cc:2229] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:17.018386 24047 raft_consensus.cc:2258] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:17.018831 24047 ts_tablet_manager.cc:1875] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:17.021361 24047 ts_tablet_manager.cc:1888] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:17.021528 24047 log.cc:1196] T 87d356a7ac32469c824af0970730214f P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/87d356a7ac32469c824af0970730214f
I20240426 09:50:17.022279 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 87d356a7ac32469c824af0970730214f (table rebalancer_extra_table_54 [id=999dc20a4b2c4d8fa90669ee871437e6]) successfully deleted
I20240426 09:50:17.150493 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:17.152045 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:17.157023 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:17.157238 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:17.162995 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:17.174966 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:17.184028 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:17.209638 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:17.257278 9669 consensus_queue.cc:237] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:17.258468 7143 raft_consensus.cc:1261] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:17.258467 9284 raft_consensus.cc:1261] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:17.258842 21826 consensus_queue.cc:1035] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:17.259045 23687 consensus_queue.cc:1035] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:17.261003 21826 raft_consensus.cc:2932] T 360aa06557974c5ea635a32245c0903e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:17.261322 7143 raft_consensus.cc:2932] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:17.263182 9284 raft_consensus.cc:2932] T 360aa06557974c5ea635a32245c0903e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:17.265841 6734 catalog_manager.cc:5484] T 360aa06557974c5ea635a32245c0903e P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:17.271790 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 360aa06557974c5ea635a32245c0903e with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:17.272194 24113 tablet_replica.cc:330] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:17.272352 24113 raft_consensus.cc:2229] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:17.272547 24113 raft_consensus.cc:2258] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:17.273063 24113 ts_tablet_manager.cc:1875] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:17.275621 24113 ts_tablet_manager.cc:1888] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:17.275696 24113 log.cc:1196] T 360aa06557974c5ea635a32245c0903e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/360aa06557974c5ea635a32245c0903e
I20240426 09:50:17.276350 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 360aa06557974c5ea635a32245c0903e (table rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]) successfully deleted
I20240426 09:50:17.411584 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:17.417577 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:17.419440 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:17.440446 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:17.441500 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:17.443698 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:17.446339 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:17.452365 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:17.524933 7013 consensus_queue.cc:237] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:17.526191 9417 raft_consensus.cc:1261] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:17.526532 7143 raft_consensus.cc:1261] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:17.526659 23729 consensus_queue.cc:1035] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:17.526917 24030 consensus_queue.cc:1035] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:17.528762 24030 raft_consensus.cc:2932] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:17.530472 7143 raft_consensus.cc:2932] T 10118b6d060f4999a2cdfdaa1b8de197 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:17.532187 6731 catalog_manager.cc:5484] T 10118b6d060f4999a2cdfdaa1b8de197 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:17.533618 9417 raft_consensus.cc:2932] T 10118b6d060f4999a2cdfdaa1b8de197 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:17.535135 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 10118b6d060f4999a2cdfdaa1b8de197 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:17.541430 24180 tablet_replica.cc:330] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:17.541682 24180 raft_consensus.cc:2229] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:17.541890 24180 raft_consensus.cc:2258] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:17.544008 24180 ts_tablet_manager.cc:1875] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:17.547461 24180 ts_tablet_manager.cc:1888] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:17.547598 24180 log.cc:1196] T 10118b6d060f4999a2cdfdaa1b8de197 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/10118b6d060f4999a2cdfdaa1b8de197
I20240426 09:50:17.548211 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 10118b6d060f4999a2cdfdaa1b8de197 (table rebalancer_extra_table_6 [id=007b1bb21a23440abb17fe4f7e152697]) successfully deleted
I20240426 09:50:17.693262 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:17.705796 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:17.705813 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:17.708380 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:17.709774 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:17.716234 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:17.716892 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:17.721436 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:17.802178 7013 consensus_queue.cc:237] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:17.803457 9546 raft_consensus.cc:1261] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:17.803808 6881 raft_consensus.cc:1261] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:17.804198 23729 consensus_queue.cc:1035] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:17.804435 24030 consensus_queue.cc:1035] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:17.806237 24257 raft_consensus.cc:2932] T 5f5418f85c2b49ce89f82331689e1b30 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:17.806432 9546 raft_consensus.cc:2932] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:17.806519 6881 raft_consensus.cc:2932] T 5f5418f85c2b49ce89f82331689e1b30 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:17.810436 6737 catalog_manager.cc:5484] T 5f5418f85c2b49ce89f82331689e1b30 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:17.815268 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f5418f85c2b49ce89f82331689e1b30 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:17.815850 24263 tablet_replica.cc:330] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:17.816012 24263 raft_consensus.cc:2229] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:17.816201 24263 raft_consensus.cc:2258] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:17.816651 24263 ts_tablet_manager.cc:1875] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:17.818606 24263 ts_tablet_manager.cc:1888] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:17.818681 24263 log.cc:1196] T 5f5418f85c2b49ce89f82331689e1b30 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/5f5418f85c2b49ce89f82331689e1b30
I20240426 09:50:17.819264 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 5f5418f85c2b49ce89f82331689e1b30 (table rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]) successfully deleted
I20240426 09:50:17.976912 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:17.979142 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:17.984539 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:17.998809 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:18.007040 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:18.012766 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:18.015602 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:18.064111 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:18.114053 9546 consensus_queue.cc:237] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:18.115448 9286 raft_consensus.cc:1261] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:50:18.115447 7011 raft_consensus.cc:1261] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:50:18.115849 23372 consensus_queue.cc:1035] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:18.116043 23032 consensus_queue.cc:1035] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:18.118078 23032 raft_consensus.cc:2932] T 8c623acd7cc9480faec733d2fd109117 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:18.118510 9286 raft_consensus.cc:2932] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:18.118851 7011 raft_consensus.cc:2932] T 8c623acd7cc9480faec733d2fd109117 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:18.122506 6737 catalog_manager.cc:5484] T 8c623acd7cc9480faec733d2fd109117 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 2 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 2 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:18.126291 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c623acd7cc9480faec733d2fd109117 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:18.126578 24263 tablet_replica.cc:330] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:18.126731 24263 raft_consensus.cc:2229] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:18.126878 24263 raft_consensus.cc:2258] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:18.127373 24263 ts_tablet_manager.cc:1875] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:18.129364 24263 ts_tablet_manager.cc:1888] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240426 09:50:18.129416 24263 log.cc:1196] T 8c623acd7cc9480faec733d2fd109117 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/8c623acd7cc9480faec733d2fd109117
I20240426 09:50:18.129920 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 8c623acd7cc9480faec733d2fd109117 (table rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]) successfully deleted
I20240426 09:50:18.280915 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:18.281149 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:18.284323 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:18.290582 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:18.304067 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:18.319013 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:18.327455 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:18.343351 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:18.405727 9286 consensus_queue.cc:237] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:18.406908 9672 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240426 09:50:18.407039 7274 raft_consensus.cc:1261] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240426 09:50:18.407320 24010 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240426 09:50:18.407523 24414 consensus_queue.cc:1035] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240426 09:50:18.409178 24010 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.409598 9672 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.409904 7274 raft_consensus.cc:2932] T 5fe43b0765d9401a88c8e9a028ab3ef4 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 6 to 8, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.413257 6740 catalog_manager.cc:5484] T 5fe43b0765d9401a88c8e9a028ab3ef4 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 6 to 8, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New cstate: current_term: 2 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.416124 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fe43b0765d9401a88c8e9a028ab3ef4 with delete_type TABLET_DATA_TOMBSTONED (TS c244e02f3cf847c0a5b69e1059f1071c not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:44196
I20240426 09:50:18.416519 24433 tablet_replica.cc:330] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:50:18.416723 24433 raft_consensus.cc:2229] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:18.416873 24433 raft_consensus.cc:2258] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:18.417778 24433 ts_tablet_manager.cc:1875] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:18.420351 24433 ts_tablet_manager.cc:1888] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20240426 09:50:18.420433 24433 log.cc:1196] T 5fe43b0765d9401a88c8e9a028ab3ef4 P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/5fe43b0765d9401a88c8e9a028ab3ef4
I20240426 09:50:18.421006 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 5fe43b0765d9401a88c8e9a028ab3ef4 (table rebalance_test_table_15 [id=755e5971be8c4ae5aaf8f1d0bb8bc16e]) successfully deleted
I20240426 09:50:18.483408 9286 consensus_queue.cc:237] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
W20240426 09:50:18.485002 9221 consensus_peers.cc:485] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d1d91861708146eb94dcc2cdef1517eb. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:18.485142 9548 raft_consensus.cc:1261] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:18.485291 9415 raft_consensus.cc:1261] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:18.485576 24414 consensus_queue.cc:1035] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:18.485783 24330 consensus_queue.cc:1035] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:18.488088 24330 raft_consensus.cc:2932] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:18.488433 9548 raft_consensus.cc:2932] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:18.490221 9415 raft_consensus.cc:2932] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:18.492048 6734 catalog_manager.cc:5484] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:18.515118 7142 consensus_queue.cc:237] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:18.516744 7013 raft_consensus.cc:1261] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:18.516779 6880 raft_consensus.cc:1261] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:18.517066 7078 consensus_peers.cc:485] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 63f066721f3441299fe5dffd07c1afed. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:18.517364 22309 consensus_queue.cc:1035] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:18.517629 24197 consensus_queue.cc:1035] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:18.520076 22309 raft_consensus.cc:2932] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.522274 7013 raft_consensus.cc:2932] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.522338 6880 raft_consensus.cc:2932] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.524441 6740 catalog_manager.cc:5484] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 4, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.553254 9669 consensus_queue.cc:237] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:18.554801 9286 raft_consensus.cc:1261] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:18.554802 9415 raft_consensus.cc:1261] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:18.555217 24107 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:18.555388 9614 consensus_peers.cc:485] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6112d2390c994fefb74b8e7f75d57d17. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:18.555516 24524 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:18.557312 24107 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:18.557752 9286 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:18.559346 9415 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:18.561199 6734 catalog_manager.cc:5484] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:18.588367 7142 consensus_queue.cc:237] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:18.589866 9546 raft_consensus.cc:1261] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:18.589877 9669 raft_consensus.cc:1261] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:18.590451 7079 consensus_peers.cc:485] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 981b71e5e8b94db7b909ec0b7564152a. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:18.590559 22309 consensus_queue.cc:1035] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:18.590744 24197 consensus_queue.cc:1035] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:18.594405 24197 raft_consensus.cc:2932] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:18.594821 9546 raft_consensus.cc:2932] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:18.597306 9669 raft_consensus.cc:2932] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:18.598448 6740 catalog_manager.cc:5484] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:18.621979 6880 consensus_queue.cc:237] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
W20240426 09:50:18.623600 6816 consensus_peers.cc:485] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1fa5a98645834e2886ff45b8b28c1c91. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:18.623795 9546 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:18.623853 7142 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240426 09:50:18.624256 24413 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:18.624621 22693 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:18.631582 22693 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.633841 7142 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.636855 6735 catalog_manager.cc:5484] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.639063 9546 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:18.771430 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:18.778257 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:18.778514 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:18.779106 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:18.780138 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:18.780632 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:18.782123 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:18.798914 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:18.991916 24662 ts_tablet_manager.cc:902] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:18.992645 24662 tablet_copy_client.cc:308] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:18.993201 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet d1d91861708146eb94dcc2cdef1517eb from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:38007)
I20240426 09:50:18.993327 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet d1d91861708146eb94dcc2cdef1517eb from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:38007: session id = e3394a4be9ea4def8aa39b333c82c5ff-d1d91861708146eb94dcc2cdef1517eb
I20240426 09:50:18.994279 9306 tablet_copy_source_session.cc:205] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:18.996153 24662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d91861708146eb94dcc2cdef1517eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:18.999023 24662 tablet_copy_client.cc:783] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:18.999190 24662 tablet_copy_client.cc:647] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:19.002595 24662 tablet_copy_client.cc:523] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:19.002647 24678 ts_tablet_manager.cc:902] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:19.003445 24678 tablet_copy_client.cc:308] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:19.004103 24662 tablet_bootstrap.cc:492] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:19.005399 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 1fa5a98645834e2886ff45b8b28c1c91 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:54839)
I20240426 09:50:19.005519 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 1fa5a98645834e2886ff45b8b28c1c91 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:54839: session id = a692602e1a2e4794877e6fb92a7b9962-1fa5a98645834e2886ff45b8b28c1c91
I20240426 09:50:19.006502 6902 tablet_copy_source_session.cc:205] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:19.007400 24678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa5a98645834e2886ff45b8b28c1c91. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:19.009693 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:19.011739 24678 tablet_copy_client.cc:783] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:19.011912 24678 tablet_copy_client.cc:647] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:19.017776 24662 tablet_bootstrap.cc:492] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:19.018040 24662 tablet_bootstrap.cc:492] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:19.018208 24662 ts_tablet_manager.cc:1367] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.001s
I20240426 09:50:19.018736 24662 raft_consensus.cc:348] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:19.018879 24662 raft_consensus.cc:729] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:19.019029 24662 consensus_queue.cc:260] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:19.019371 24662 ts_tablet_manager.cc:1398] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:19.023972 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-d1d91861708146eb94dcc2cdef1517eb received from {username='slave'} at 127.0.99.132:38007
I20240426 09:50:19.024066 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-d1d91861708146eb94dcc2cdef1517eb on tablet d1d91861708146eb94dcc2cdef1517eb with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:19.026789 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:19.028339 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:19.032256 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:19.045436 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:19.050662 24678 tablet_copy_client.cc:523] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:19.052434 24678 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:19.061982 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:19.066859 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:19.069273 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:19.077491 24685 ts_tablet_manager.cc:902] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:19.078063 24685 tablet_copy_client.cc:308] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:19.078485 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 6112d2390c994fefb74b8e7f75d57d17 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:46939)
I20240426 09:50:19.078581 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 6112d2390c994fefb74b8e7f75d57d17 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:46939: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-6112d2390c994fefb74b8e7f75d57d17
I20240426 09:50:19.079525 9685 tablet_copy_source_session.cc:205] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:19.080477 24685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6112d2390c994fefb74b8e7f75d57d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:19.086848 24685 tablet_copy_client.cc:783] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:19.087024 24685 tablet_copy_client.cc:647] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:19.087376 24678 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:19.087695 24678 tablet_bootstrap.cc:492] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:19.087888 24678 ts_tablet_manager.cc:1367] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.036s user 0.026s sys 0.003s
I20240426 09:50:19.088335 24678 raft_consensus.cc:348] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:19.088496 24678 raft_consensus.cc:729] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:19.088642 24678 consensus_queue.cc:260] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:19.089016 24678 ts_tablet_manager.cc:1398] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:50:19.089509 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-1fa5a98645834e2886ff45b8b28c1c91 received from {username='slave'} at 127.0.99.133:54839
I20240426 09:50:19.089586 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-1fa5a98645834e2886ff45b8b28c1c91 on tablet 1fa5a98645834e2886ff45b8b28c1c91 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:19.096741 24678 ts_tablet_manager.cc:902] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:19.099608 24678 tablet_copy_client.cc:308] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:19.100030 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 63f066721f3441299fe5dffd07c1afed from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:54629)
I20240426 09:50:19.100129 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 63f066721f3441299fe5dffd07c1afed from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:54629: session id = a692602e1a2e4794877e6fb92a7b9962-63f066721f3441299fe5dffd07c1afed
I20240426 09:50:19.101047 7164 tablet_copy_source_session.cc:205] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:19.101938 24678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f066721f3441299fe5dffd07c1afed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:19.102185 24685 tablet_copy_client.cc:523] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:19.103842 24685 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:50:19.105619 24678 tablet_copy_client.cc:783] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:19.105865 24678 tablet_copy_client.cc:647] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:19.107326 24685 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:19.107590 24685 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:50:19.107730 24685 ts_tablet_manager.cc:1367] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240426 09:50:19.108136 24685 raft_consensus.cc:348] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:19.108247 24685 raft_consensus.cc:729] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:50:19.108654 24685 consensus_queue.cc:260] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:19.108940 24678 tablet_copy_client.cc:523] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:19.109887 24685 ts_tablet_manager.cc:1398] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:19.110285 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-6112d2390c994fefb74b8e7f75d57d17 received from {username='slave'} at 127.0.99.135:46939
I20240426 09:50:19.110363 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-6112d2390c994fefb74b8e7f75d57d17 on tablet 6112d2390c994fefb74b8e7f75d57d17 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:50:19.110436 24678 tablet_bootstrap.cc:492] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:19.139459 24678 tablet_bootstrap.cc:492] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:19.139755 24678 tablet_bootstrap.cc:492] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:19.139926 24678 ts_tablet_manager.cc:1367] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.030s user 0.023s sys 0.000s
I20240426 09:50:19.140339 24678 raft_consensus.cc:348] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:19.140446 24678 raft_consensus.cc:729] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:19.140568 24678 consensus_queue.cc:260] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:19.140857 24678 ts_tablet_manager.cc:1398] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:19.141433 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-63f066721f3441299fe5dffd07c1afed received from {username='slave'} at 127.0.99.133:54629
I20240426 09:50:19.141522 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-63f066721f3441299fe5dffd07c1afed on tablet 63f066721f3441299fe5dffd07c1afed with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:19.198479 24662 ts_tablet_manager.cc:902] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:19.198997 24662 tablet_copy_client.cc:308] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:19.199604 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 981b71e5e8b94db7b909ec0b7564152a from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:38167)
I20240426 09:50:19.199720 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 981b71e5e8b94db7b909ec0b7564152a from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:38167: session id = e3394a4be9ea4def8aa39b333c82c5ff-981b71e5e8b94db7b909ec0b7564152a
I20240426 09:50:19.200806 7164 tablet_copy_source_session.cc:205] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:19.201632 24662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 981b71e5e8b94db7b909ec0b7564152a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:19.205441 24662 tablet_copy_client.cc:783] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:19.206372 24662 tablet_copy_client.cc:647] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:19.209122 24662 tablet_copy_client.cc:523] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:19.212080 24662 tablet_bootstrap.cc:492] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:19.225323 24662 tablet_bootstrap.cc:492] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:19.225641 24662 tablet_bootstrap.cc:492] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:19.225816 24662 ts_tablet_manager.cc:1367] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.002s
I20240426 09:50:19.226270 24662 raft_consensus.cc:348] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:19.226377 24662 raft_consensus.cc:729] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:19.226548 24662 consensus_queue.cc:260] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:19.226893 24662 ts_tablet_manager.cc:1398] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:19.227349 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-981b71e5e8b94db7b909ec0b7564152a received from {username='slave'} at 127.0.99.132:38167
I20240426 09:50:19.227424 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-981b71e5e8b94db7b909ec0b7564152a on tablet 981b71e5e8b94db7b909ec0b7564152a with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:19.280473 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:19.293138 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:19.296209 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:19.296206 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:19.296664 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:19.299893 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:19.312901 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:19.313445 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:19.460961 9548 raft_consensus.cc:1203] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:19.531940 7275 raft_consensus.cc:1203] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:19.550133 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:19.553576 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:19.557423 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:19.559044 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:19.564195 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:19.564447 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:19.565413 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:19.577667 9284 raft_consensus.cc:1203] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240426 09:50:19.592228 9284 raft_consensus.cc:1203] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:19.603230 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:19.674607 9284 consensus_queue.cc:237] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:19.675895 9548 raft_consensus.cc:1261] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:19.675998 7275 raft_consensus.cc:1261] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:19.676317 24330 consensus_queue.cc:1035] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:19.676462 24414 consensus_queue.cc:1035] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:19.678158 24820 raft_consensus.cc:2932] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:19.678509 7275 raft_consensus.cc:2932] T d1d91861708146eb94dcc2cdef1517eb P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:19.678543 9548 raft_consensus.cc:2932] T d1d91861708146eb94dcc2cdef1517eb P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:19.682001 6737 catalog_manager.cc:5484] T d1d91861708146eb94dcc2cdef1517eb P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:19.686529 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet d1d91861708146eb94dcc2cdef1517eb with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:19.687340 24830 tablet_replica.cc:330] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:19.687603 24830 raft_consensus.cc:2229] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:19.687799 24830 raft_consensus.cc:2258] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:19.688311 24830 ts_tablet_manager.cc:1875] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:19.690649 24830 ts_tablet_manager.cc:1888] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:19.690732 24830 log.cc:1196] T d1d91861708146eb94dcc2cdef1517eb P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/d1d91861708146eb94dcc2cdef1517eb
I20240426 09:50:19.691296 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet d1d91861708146eb94dcc2cdef1517eb (table rebalancer_extra_table_32 [id=c97fc87e609640c1b59fee65f9beb869]) successfully deleted
I20240426 09:50:19.746359 7275 raft_consensus.cc:1203] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:19.816646 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:19.817548 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:19.831810 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:19.833650 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:19.834113 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:19.861128 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:19.869581 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:19.881645 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:19.940129 6880 consensus_queue.cc:237] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:19.941546 9286 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:19.941561 9546 raft_consensus.cc:1261] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240426 09:50:19.942034 24562 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:19.942553 24562 consensus_queue.cc:1035] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:19.943734 24562 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:19.944204 9284 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:19.947568 6731 catalog_manager.cc:5484] T 1fa5a98645834e2886ff45b8b28c1c91 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 6 to 7, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:19.949404 9546 raft_consensus.cc:2932] T 1fa5a98645834e2886ff45b8b28c1c91 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:19.950938 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa5a98645834e2886ff45b8b28c1c91 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:19.959194 24897 tablet_replica.cc:330] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:19.959370 24897 raft_consensus.cc:2229] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:19.959574 24897 raft_consensus.cc:2258] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:19.960290 24897 ts_tablet_manager.cc:1875] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:19.963505 24897 ts_tablet_manager.cc:1888] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240426 09:50:19.963629 24897 log.cc:1196] T 1fa5a98645834e2886ff45b8b28c1c91 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/1fa5a98645834e2886ff45b8b28c1c91
I20240426 09:50:19.964255 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 1fa5a98645834e2886ff45b8b28c1c91 (table rebalance_test_table_7 [id=52ede4d5461a4014b73c57ffee2e6b35]) successfully deleted
I20240426 09:50:20.107424 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:20.114485 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:20.116495 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:20.123363 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:20.123380 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:20.134647 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:20.141234 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:20.151824 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:20.226199 7011 consensus_queue.cc:237] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:20.227463 7275 raft_consensus.cc:1261] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:20.227552 6880 raft_consensus.cc:1261] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:20.227880 24030 consensus_queue.cc:1035] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:20.228089 24684 consensus_queue.cc:1035] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:20.230168 24684 raft_consensus.cc:2932] T a31b0f75c2514916a94205ce0289629e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:20.230618 7275 raft_consensus.cc:2932] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:20.232651 6880 raft_consensus.cc:2932] T a31b0f75c2514916a94205ce0289629e P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:20.234022 6740 catalog_manager.cc:5484] T a31b0f75c2514916a94205ce0289629e P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:20.237396 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet a31b0f75c2514916a94205ce0289629e with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:20.237771 24972 tablet_replica.cc:330] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:20.237932 24972 raft_consensus.cc:2229] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:20.238180 24972 raft_consensus.cc:2258] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:20.238631 24972 ts_tablet_manager.cc:1875] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:20.241173 24972 ts_tablet_manager.cc:1888] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:20.241307 24972 log.cc:1196] T a31b0f75c2514916a94205ce0289629e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/a31b0f75c2514916a94205ce0289629e
I20240426 09:50:20.241907 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet a31b0f75c2514916a94205ce0289629e (table rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]) successfully deleted
I20240426 09:50:20.363750 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:20.373277 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:20.374936 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:20.381532 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:20.382953 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:20.389094 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:20.401396 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:20.410609 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:20.472385 6880 consensus_queue.cc:237] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:20.473901 7275 raft_consensus.cc:1261] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:20.474035 9417 raft_consensus.cc:1261] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:20.474325 24562 consensus_queue.cc:1035] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:20.474519 24896 consensus_queue.cc:1035] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:20.476397 25043 raft_consensus.cc:2932] T 538173a16ff94e70bc8d25b8f1e7c84e P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:20.476706 9417 raft_consensus.cc:2932] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:20.476749 7275 raft_consensus.cc:2932] T 538173a16ff94e70bc8d25b8f1e7c84e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:20.480854 6738 catalog_manager.cc:5484] T 538173a16ff94e70bc8d25b8f1e7c84e P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:20.484529 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 538173a16ff94e70bc8d25b8f1e7c84e with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:20.484750 24972 tablet_replica.cc:330] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:20.484910 24972 raft_consensus.cc:2229] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:20.485054 24972 raft_consensus.cc:2258] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:20.485460 24972 ts_tablet_manager.cc:1875] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:20.487704 24972 ts_tablet_manager.cc:1888] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:20.487782 24972 log.cc:1196] T 538173a16ff94e70bc8d25b8f1e7c84e P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/538173a16ff94e70bc8d25b8f1e7c84e
I20240426 09:50:20.488281 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 538173a16ff94e70bc8d25b8f1e7c84e (table rebalancer_extra_table_34 [id=150bf96c79394f67906b8dd1a2a7605b]) successfully deleted
I20240426 09:50:20.631408 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:20.638068 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:20.638700 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:20.648801 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:20.650595 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:20.660821 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:20.673833 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:20.676602 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:20.733481 9672 consensus_queue.cc:237] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:20.734624 9284 raft_consensus.cc:1261] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:20.734663 9546 raft_consensus.cc:1261] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:20.735059 24527 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:20.735239 24524 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:20.737047 25114 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:20.737279 9546 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:20.737314 9284 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:20.741585 6739 catalog_manager.cc:5484] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:20.744964 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 6112d2390c994fefb74b8e7f75d57d17 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:20.745640 25115 tablet_replica.cc:330] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:20.745852 25115 raft_consensus.cc:2229] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:20.746033 25115 raft_consensus.cc:2258] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:20.746533 25115 ts_tablet_manager.cc:1875] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:20.749802 25115 ts_tablet_manager.cc:1888] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:20.749886 25115 log.cc:1196] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/6112d2390c994fefb74b8e7f75d57d17
I20240426 09:50:20.750509 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 6112d2390c994fefb74b8e7f75d57d17 (table rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]) successfully deleted
I20240426 09:50:20.883824 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:20.884338 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:20.885177 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:20.886469 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:20.890344 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:20.916496 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:20.931687 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:20.949072 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:21.012194 7011 consensus_queue.cc:237] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:21.013556 9672 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240426 09:50:21.013674 9286 raft_consensus.cc:1261] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240426 09:50:21.013976 24969 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240426 09:50:21.014405 24969 consensus_queue.cc:1035] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240426 09:50:21.016650 24969 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 7 to 8, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:21.017076 9669 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:21.020707 6739 catalog_manager.cc:5484] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 7 to 8, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:21.022572 9284 raft_consensus.cc:2932] T 643fd2d359a14f8eaa8d8a0d0879ef61 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:21.028709 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 643fd2d359a14f8eaa8d8a0d0879ef61 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:21.029536 25193 tablet_replica.cc:330] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:21.029798 25193 raft_consensus.cc:2229] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:21.030036 25193 raft_consensus.cc:2258] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:21.031764 25193 ts_tablet_manager.cc:1875] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:21.034691 25193 ts_tablet_manager.cc:1888] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.7
I20240426 09:50:21.034850 25193 log.cc:1196] T 643fd2d359a14f8eaa8d8a0d0879ef61 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/643fd2d359a14f8eaa8d8a0d0879ef61
I20240426 09:50:21.035456 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 643fd2d359a14f8eaa8d8a0d0879ef61 (table rebalance_test_table_13 [id=3a872d0f00264ee6962347b7362cd103]) successfully deleted
I20240426 09:50:21.099133 7274 consensus_queue.cc:237] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:21.100818 9417 raft_consensus.cc:1261] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:21.100905 7209 consensus_peers.cc:485] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c6956266cb444536945540ecdbaba28e. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:21.101229 7011 raft_consensus.cc:1261] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:21.101282 25235 consensus_queue.cc:1035] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:21.101601 25132 consensus_queue.cc:1035] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:21.103420 25132 raft_consensus.cc:2932] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:21.103826 9417 raft_consensus.cc:2932] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:21.106288 7011 raft_consensus.cc:2932] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:21.108155 6739 catalog_manager.cc:5484] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:21.129864 6881 consensus_queue.cc:237] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:21.131448 9669 raft_consensus.cc:1261] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:21.131594 6815 consensus_peers.cc:485] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b5a4a67b5ab943b0a6299d74f2411190. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:21.131789 25254 consensus_queue.cc:1035] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:21.131836 9285 raft_consensus.cc:1261] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:21.132156 24896 consensus_queue.cc:1035] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:21.135290 25254 raft_consensus.cc:2932] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:21.135828 9285 raft_consensus.cc:2932] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:21.137972 9669 raft_consensus.cc:2932] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:21.139734 6734 catalog_manager.cc:5484] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:21.160857 9417 consensus_queue.cc:237] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:21.162523 7142 raft_consensus.cc:1261] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:21.162598 9352 consensus_peers.cc:485] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b017fe28f5f241af8955642664afaaa8. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:21.162987 24453 consensus_queue.cc:1035] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:21.163113 9546 raft_consensus.cc:1261] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:21.163583 24453 consensus_queue.cc:1035] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:21.165591 25273 raft_consensus.cc:2932] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:21.165963 9546 raft_consensus.cc:2932] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:21.167469 7142 raft_consensus.cc:2932] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:21.170020 6739 catalog_manager.cc:5484] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:21.311219 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:21.312033 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:21.312041 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:21.316602 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:21.325304 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:21.341997 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:21.342504 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:21.342804 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:21.531145 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:21.533210 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:21.537127 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:21.539978 25383 ts_tablet_manager.cc:902] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:21.540519 25383 tablet_copy_client.cc:308] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:21.541512 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet b017fe28f5f241af8955642664afaaa8 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:53025)
I20240426 09:50:21.541606 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet b017fe28f5f241af8955642664afaaa8 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:53025: session id = e3394a4be9ea4def8aa39b333c82c5ff-b017fe28f5f241af8955642664afaaa8
I20240426 09:50:21.542672 9437 tablet_copy_source_session.cc:205] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:21.544361 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:21.550870 25382 ts_tablet_manager.cc:902] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:21.551203 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:21.554665 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:21.559139 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:21.551395 25382 tablet_copy_client.cc:308] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:21.560002 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet b5a4a67b5ab943b0a6299d74f2411190 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:53629)
I20240426 09:50:21.560139 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet b5a4a67b5ab943b0a6299d74f2411190 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:53629: session id = 1aaf8befba984e199e96f53f46b50441-b5a4a67b5ab943b0a6299d74f2411190
I20240426 09:50:21.561224 6902 tablet_copy_source_session.cc:205] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:21.562484 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:21.566008 25382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5a4a67b5ab943b0a6299d74f2411190. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:21.587225 25388 ts_tablet_manager.cc:902] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:50:21.602530 25388 tablet_copy_client.cc:308] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:50:21.603035 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet c6956266cb444536945540ecdbaba28e from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:40185)
I20240426 09:50:21.603158 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet c6956266cb444536945540ecdbaba28e from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:40185: session id = a692602e1a2e4794877e6fb92a7b9962-c6956266cb444536945540ecdbaba28e
I20240426 09:50:21.604120 7295 tablet_copy_source_session.cc:205] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:21.608310 25388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6956266cb444536945540ecdbaba28e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:21.611538 25388 tablet_copy_client.cc:783] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:21.611690 25388 tablet_copy_client.cc:647] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:21.617650 25383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b017fe28f5f241af8955642664afaaa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:21.622828 25383 tablet_copy_client.cc:783] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:21.622997 25383 tablet_copy_client.cc:647] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:21.635967 25388 tablet_copy_client.cc:523] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:21.636531 25383 tablet_copy_client.cc:523] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:21.637532 25388 tablet_bootstrap.cc:492] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:21.639521 25383 tablet_bootstrap.cc:492] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:21.640331 25382 tablet_copy_client.cc:783] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:21.640509 25382 tablet_copy_client.cc:647] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:21.641719 25382 tablet_copy_client.cc:523] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:21.643240 25382 tablet_bootstrap.cc:492] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:21.646139 25383 tablet_bootstrap.cc:492] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:21.646446 25383 tablet_bootstrap.cc:492] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:21.646624 25383 ts_tablet_manager.cc:1367] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240426 09:50:21.647598 25383 raft_consensus.cc:348] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:21.649255 25383 raft_consensus.cc:729] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:21.649394 25383 consensus_queue.cc:260] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:21.649792 25383 ts_tablet_manager.cc:1398] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:50:21.650197 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-b017fe28f5f241af8955642664afaaa8 received from {username='slave'} at 127.0.99.132:53025
I20240426 09:50:21.650267 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-b017fe28f5f241af8955642664afaaa8 on tablet b017fe28f5f241af8955642664afaaa8 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:21.656172 25382 tablet_bootstrap.cc:492] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:21.656673 25382 tablet_bootstrap.cc:492] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:21.656844 25382 ts_tablet_manager.cc:1367] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.002s
I20240426 09:50:21.657440 25382 raft_consensus.cc:348] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:21.657562 25382 raft_consensus.cc:729] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:21.657701 25382 consensus_queue.cc:260] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:21.658051 25382 ts_tablet_manager.cc:1398] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:21.658488 25388 tablet_bootstrap.cc:492] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:21.658733 25388 tablet_bootstrap.cc:492] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:21.660157 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-b5a4a67b5ab943b0a6299d74f2411190 received from {username='slave'} at 127.0.99.131:53629
I20240426 09:50:21.660243 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-b5a4a67b5ab943b0a6299d74f2411190 on tablet b5a4a67b5ab943b0a6299d74f2411190 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:21.661150 25388 ts_tablet_manager.cc:1367] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20240426 09:50:21.661629 25388 raft_consensus.cc:348] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:21.661747 25388 raft_consensus.cc:729] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:21.661867 25388 consensus_queue.cc:260] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:21.662175 25388 ts_tablet_manager.cc:1398] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:21.662705 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-c6956266cb444536945540ecdbaba28e received from {username='slave'} at 127.0.99.133:40185
I20240426 09:50:21.662828 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-c6956266cb444536945540ecdbaba28e on tablet c6956266cb444536945540ecdbaba28e with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:21.799119 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:21.803893 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:21.815883 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:21.819802 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:21.819802 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:21.820626 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:21.820894 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:21.825333 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:21.908473 7143 consensus_queue.cc:237] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:21.909801 7275 raft_consensus.cc:1261] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:21.909801 9669 raft_consensus.cc:1261] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:21.910279 22309 consensus_queue.cc:1035] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:21.910471 24331 consensus_queue.cc:1035] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:21.913121 24331 raft_consensus.cc:2932] T 981b71e5e8b94db7b909ec0b7564152a P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:21.913657 7274 raft_consensus.cc:2932] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:21.914956 9669 raft_consensus.cc:2932] T 981b71e5e8b94db7b909ec0b7564152a P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:21.916720 6734 catalog_manager.cc:5484] T 981b71e5e8b94db7b909ec0b7564152a P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:21.922025 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 981b71e5e8b94db7b909ec0b7564152a with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:21.923154 25481 tablet_replica.cc:330] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:21.923540 25481 raft_consensus.cc:2229] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:21.923820 25481 raft_consensus.cc:2258] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:21.924228 25481 ts_tablet_manager.cc:1875] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:21.928772 25481 ts_tablet_manager.cc:1888] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:21.928845 25481 log.cc:1196] T 981b71e5e8b94db7b909ec0b7564152a P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/981b71e5e8b94db7b909ec0b7564152a
I20240426 09:50:21.929360 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 981b71e5e8b94db7b909ec0b7564152a (table rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]) successfully deleted
I20240426 09:50:21.933300 9284 raft_consensus.cc:1203] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:21.966311 7274 raft_consensus.cc:1203] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:22.082649 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:22.086496 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:22.095677 7123 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240426 09:50:22.108965 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:22.109755 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:22.121479 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:22.124421 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:22.124785 7143 raft_consensus.cc:1203] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:22.125808 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:22.201377 7143 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "63f066721f3441299fe5dffd07c1afed"
dest_uuid: "1aaf8befba984e199e96f53f46b50441"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37854
I20240426 09:50:22.201519 7143 raft_consensus.cc:595] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:22.239801 25472 raft_consensus.cc:979] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: : Instructing follower 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:50:22.239914 25472 raft_consensus.cc:1067] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Signalling peer 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:50:22.240607 7013 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "63f066721f3441299fe5dffd07c1afed"
dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
from {username='slave'} at 127.0.99.131:54895
I20240426 09:50:22.240744 7013 raft_consensus.cc:482] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:22.240809 7013 raft_consensus.cc:3037] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:22.242091 7013 raft_consensus.cc:504] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:22.242594 7013 leader_election.cc:288] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:22.242952 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f066721f3441299fe5dffd07c1afed" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:50:22.242961 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f066721f3441299fe5dffd07c1afed" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:50:22.243080 9284 raft_consensus.cc:3037] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:22.243088 7143 raft_consensus.cc:3032] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:22.243134 7143 raft_consensus.cc:729] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:50:22.243301 7143 consensus_queue.cc:260] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:22.243518 7143 raft_consensus.cc:3037] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:22.243628 6881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f066721f3441299fe5dffd07c1afed" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:50:22.243750 6881 raft_consensus.cc:3037] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:22.244125 9284 raft_consensus.cc:2454] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:50:22.244601 7143 raft_consensus.cc:2454] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:50:22.244732 6881 raft_consensus.cc:2454] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:50:22.244906 6946 leader_election.cc:302] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441, a692602e1a2e4794877e6fb92a7b9962; no voters:
I20240426 09:50:22.245128 25187 raft_consensus.cc:2781] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:22.245232 25187 raft_consensus.cc:686] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:50:22.245390 25187 consensus_queue.cc:237] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:22.252054 6736 catalog_manager.cc:5484] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 1 to 2, leader changed from 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:22.329774 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:22.331997 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:22.332800 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:22.335008 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:22.340472 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:22.343469 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:22.368489 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:22.388981 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:22.556324 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:22.566574 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:22.566720 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:22.570950 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:22.579241 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:22.605733 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:22.612198 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:22.612558 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:22.639277 7142 raft_consensus.cc:1261] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:22.639890 25187 consensus_queue.cc:1035] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:22.642431 9285 raft_consensus.cc:1261] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:22.645869 24030 consensus_queue.cc:1035] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:22.656785 6880 raft_consensus.cc:1261] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:22.657177 24030 consensus_queue.cc:1035] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:22.695125 9417 consensus_queue.cc:237] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:22.696472 7275 raft_consensus.cc:1261] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:22.696610 7142 raft_consensus.cc:1261] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:22.697115 25417 consensus_queue.cc:1035] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:22.697324 24453 consensus_queue.cc:1035] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:22.699862 24453 raft_consensus.cc:2932] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.700373 7142 raft_consensus.cc:2932] T b017fe28f5f241af8955642664afaaa8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.700991 7275 raft_consensus.cc:2932] T b017fe28f5f241af8955642664afaaa8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.703644 6739 catalog_manager.cc:5484] T b017fe28f5f241af8955642664afaaa8 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.707121 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet b017fe28f5f241af8955642664afaaa8 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:22.707496 25663 tablet_replica.cc:330] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:22.709087 25663 raft_consensus.cc:2229] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:22.709251 25663 raft_consensus.cc:2258] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:22.709726 25663 ts_tablet_manager.cc:1875] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:22.712061 25663 ts_tablet_manager.cc:1888] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:22.712214 25663 log.cc:1196] T b017fe28f5f241af8955642664afaaa8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/b017fe28f5f241af8955642664afaaa8
I20240426 09:50:22.712728 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet b017fe28f5f241af8955642664afaaa8 (table rebalancer_extra_table_50 [id=e223c8be82ff430ca725d1d4c0a3cf58]) successfully deleted
I20240426 09:50:22.762317 7275 consensus_queue.cc:237] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:22.764088 9284 raft_consensus.cc:1261] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:22.764214 6880 raft_consensus.cc:1261] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:22.764637 25711 consensus_queue.cc:1035] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:22.764854 25529 consensus_queue.cc:1035] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:22.764997 7208 consensus_peers.cc:485] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d30be76596534071a3ea9a90375099f6. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:22.767172 25529 raft_consensus.cc:2932] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:22.767668 6880 raft_consensus.cc:2932] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:22.770185 9284 raft_consensus.cc:2932] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:22.771173 6732 catalog_manager.cc:5484] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:22.791708 9417 consensus_queue.cc:237] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:22.793176 9546 raft_consensus.cc:1261] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:22.793213 9352 consensus_peers.cc:485] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e64e5170d6bc4154b5ec869d535051c3. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:22.793282 9284 raft_consensus.cc:1261] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:22.793771 25417 consensus_queue.cc:1035] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:22.795051 25417 consensus_queue.cc:1035] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:22.797116 25417 raft_consensus.cc:2932] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.797837 9284 raft_consensus.cc:2932] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.800560 9546 raft_consensus.cc:2932] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.801446 6731 catalog_manager.cc:5484] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.823920 9546 consensus_queue.cc:237] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:22.825320 9284 raft_consensus.cc:1261] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:22.825430 6880 raft_consensus.cc:1261] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:22.825606 9482 consensus_peers.cc:485] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfe16e4114924bda8c560d4926a63923. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:22.825790 25370 consensus_queue.cc:1035] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:22.826004 25583 consensus_queue.cc:1035] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:22.827764 25370 raft_consensus.cc:2932] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:22.828202 9284 raft_consensus.cc:2932] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:22.831674 6880 raft_consensus.cc:2932] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:22.831877 6735 catalog_manager.cc:5484] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:22.852931 7142 consensus_queue.cc:237] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
W20240426 09:50:22.854614 7079 consensus_peers.cc:485] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 91a15bb5ed1541ca906b8603d9249690. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:22.854611 9284 raft_consensus.cc:1261] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:22.854657 9546 raft_consensus.cc:1261] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:22.855046 25770 consensus_queue.cc:1035] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:22.855230 22309 consensus_queue.cc:1035] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:22.857406 22309 raft_consensus.cc:2932] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.857780 9546 raft_consensus.cc:2932] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.859747 9284 raft_consensus.cc:2932] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:22.861428 6732 catalog_manager.cc:5484] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:23.003283 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:23.008399 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:23.010438 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:23.013165 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:23.029462 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:23.048432 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:23.048514 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:23.059697 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:23.242156 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:23.247443 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:23.249437 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:23.254802 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:23.258589 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:23.264180 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:23.265300 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:23.288470 25878 ts_tablet_manager.cc:902] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:23.292352 25878 tablet_copy_client.cc:308] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:50:23.292801 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet cfe16e4114924bda8c560d4926a63923 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:45949)
I20240426 09:50:23.292905 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet cfe16e4114924bda8c560d4926a63923 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:45949: session id = 1aaf8befba984e199e96f53f46b50441-cfe16e4114924bda8c560d4926a63923
I20240426 09:50:23.293781 9569 tablet_copy_source_session.cc:205] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:23.294898 25878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfe16e4114924bda8c560d4926a63923. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:23.297574 25878 tablet_copy_client.cc:783] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:23.297737 25878 tablet_copy_client.cc:647] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:23.307231 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:23.312441 25878 tablet_copy_client.cc:523] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:23.313975 25878 tablet_bootstrap.cc:492] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:23.318370 25878 tablet_bootstrap.cc:492] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:23.318627 25878 tablet_bootstrap.cc:492] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:23.318826 25878 ts_tablet_manager.cc:1367] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.006s
I20240426 09:50:23.326050 25878 raft_consensus.cc:348] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:23.326222 25878 raft_consensus.cc:729] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:23.326365 25878 consensus_queue.cc:260] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:23.326699 25878 ts_tablet_manager.cc:1398] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.008s user 0.000s sys 0.000s
I20240426 09:50:23.327157 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-cfe16e4114924bda8c560d4926a63923 received from {username='slave'} at 127.0.99.131:45949
I20240426 09:50:23.327239 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-cfe16e4114924bda8c560d4926a63923 on tablet cfe16e4114924bda8c560d4926a63923 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:23.349363 25896 ts_tablet_manager.cc:902] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:23.349902 25896 tablet_copy_client.cc:308] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:23.350482 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet e64e5170d6bc4154b5ec869d535051c3 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:53025)
I20240426 09:50:23.350594 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet e64e5170d6bc4154b5ec869d535051c3 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:53025: session id = e3394a4be9ea4def8aa39b333c82c5ff-e64e5170d6bc4154b5ec869d535051c3
I20240426 09:50:23.351658 9437 tablet_copy_source_session.cc:205] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:23.352665 25896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64e5170d6bc4154b5ec869d535051c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:23.355355 25896 tablet_copy_client.cc:783] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:23.355525 25896 tablet_copy_client.cc:647] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:23.357234 25901 ts_tablet_manager.cc:902] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:50:23.358568 25901 tablet_copy_client.cc:308] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:50:23.359196 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet d30be76596534071a3ea9a90375099f6 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:41659)
I20240426 09:50:23.359284 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet d30be76596534071a3ea9a90375099f6 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:41659: session id = 3b41d87e7bfb419f8aa7500100129b6c-d30be76596534071a3ea9a90375099f6
I20240426 09:50:23.360327 7295 tablet_copy_source_session.cc:205] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:23.360858 25896 tablet_copy_client.cc:523] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:23.361172 25901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30be76596534071a3ea9a90375099f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:23.362661 25896 tablet_bootstrap.cc:492] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:23.364188 25901 tablet_copy_client.cc:783] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:23.364356 25901 tablet_copy_client.cc:647] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:23.365355 25901 tablet_copy_client.cc:523] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:23.366495 25901 tablet_bootstrap.cc:492] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:23.368117 25896 tablet_bootstrap.cc:492] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:23.368372 25896 tablet_bootstrap.cc:492] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:23.368775 25896 ts_tablet_manager.cc:1367] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240426 09:50:23.369225 25896 raft_consensus.cc:348] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:23.369349 25896 raft_consensus.cc:729] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:23.369491 25896 consensus_queue.cc:260] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:23.369838 25896 ts_tablet_manager.cc:1398] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:23.370443 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-e64e5170d6bc4154b5ec869d535051c3 received from {username='slave'} at 127.0.99.132:53025
I20240426 09:50:23.370517 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-e64e5170d6bc4154b5ec869d535051c3 on tablet e64e5170d6bc4154b5ec869d535051c3 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:23.370685 25901 tablet_bootstrap.cc:492] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:23.370983 25901 tablet_bootstrap.cc:492] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:23.371147 25901 ts_tablet_manager.cc:1367] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20240426 09:50:23.371537 25901 raft_consensus.cc:348] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:23.371641 25901 raft_consensus.cc:729] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:23.371762 25901 consensus_queue.cc:260] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:23.372063 25901 ts_tablet_manager.cc:1398] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:23.372560 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-d30be76596534071a3ea9a90375099f6 received from {username='slave'} at 127.0.99.134:41659
I20240426 09:50:23.372634 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-d30be76596534071a3ea9a90375099f6 on tablet d30be76596534071a3ea9a90375099f6 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:23.410285 25896 ts_tablet_manager.cc:902] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:23.410831 25896 tablet_copy_client.cc:308] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:23.411310 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 91a15bb5ed1541ca906b8603d9249690 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:38167)
I20240426 09:50:23.411418 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 91a15bb5ed1541ca906b8603d9249690 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:38167: session id = e3394a4be9ea4def8aa39b333c82c5ff-91a15bb5ed1541ca906b8603d9249690
I20240426 09:50:23.412268 7164 tablet_copy_source_session.cc:205] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:23.413064 25896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91a15bb5ed1541ca906b8603d9249690. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:23.415706 25896 tablet_copy_client.cc:783] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:23.415885 25896 tablet_copy_client.cc:647] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:23.417445 25896 tablet_copy_client.cc:523] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:23.418563 25896 tablet_bootstrap.cc:492] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:23.425628 25896 tablet_bootstrap.cc:492] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:23.425901 25896 tablet_bootstrap.cc:492] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:23.426091 25896 ts_tablet_manager.cc:1367] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.001s
I20240426 09:50:23.427209 25896 raft_consensus.cc:348] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:23.427352 25896 raft_consensus.cc:729] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:23.427556 25896 consensus_queue.cc:260] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:23.428042 25896 ts_tablet_manager.cc:1398] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:23.428910 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-91a15bb5ed1541ca906b8603d9249690 received from {username='slave'} at 127.0.99.132:38167
I20240426 09:50:23.428997 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-91a15bb5ed1541ca906b8603d9249690 on tablet 91a15bb5ed1541ca906b8603d9249690 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:23.491760 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:23.499363 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:23.501053 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:23.502619 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:23.505415 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:23.505797 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:23.508108 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:23.535619 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:23.627938 6880 consensus_queue.cc:237] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:23.629312 9673 raft_consensus.cc:1261] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:23.629310 7142 raft_consensus.cc:1261] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:23.629781 24896 consensus_queue.cc:1035] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:23.630098 25636 consensus_queue.cc:1035] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:23.631680 24896 raft_consensus.cc:2932] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:23.633256 7142 raft_consensus.cc:2932] T b5a4a67b5ab943b0a6299d74f2411190 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:23.633630 9673 raft_consensus.cc:2932] T b5a4a67b5ab943b0a6299d74f2411190 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:23.636868 6732 catalog_manager.cc:5484] T b5a4a67b5ab943b0a6299d74f2411190 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:23.640517 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet b5a4a67b5ab943b0a6299d74f2411190 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:23.641060 25981 tablet_replica.cc:330] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:23.641209 25981 raft_consensus.cc:2229] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:23.641423 25981 raft_consensus.cc:2258] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:23.641973 25981 ts_tablet_manager.cc:1875] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:23.644731 25981 ts_tablet_manager.cc:1888] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:23.644899 25981 log.cc:1196] T b5a4a67b5ab943b0a6299d74f2411190 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/b5a4a67b5ab943b0a6299d74f2411190
I20240426 09:50:23.645514 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet b5a4a67b5ab943b0a6299d74f2411190 (table rebalancer_extra_table_36 [id=46b7fccd1f314598b44ec61bd100e29f]) successfully deleted
I20240426 09:50:23.768926 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:23.778736 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:23.781426 7142 raft_consensus.cc:1203] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:23.783921 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:23.786825 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:23.788959 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:23.797843 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:23.800293 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:23.802951 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:23.810284 9417 raft_consensus.cc:1203] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:23.847345 7275 raft_consensus.cc:1203] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:23.876140 7275 raft_consensus.cc:1203] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:23.994040 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:23.999199 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:24.006742 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:24.013134 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:24.014883 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:24.015779 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:24.027412 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:24.050977 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:24.110090 7275 consensus_queue.cc:237] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:24.111449 9284 raft_consensus.cc:1261] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:24.111490 7013 raft_consensus.cc:1261] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:24.112174 25711 consensus_queue.cc:1035] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:24.112526 25711 consensus_queue.cc:1035] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:24.114621 25711 raft_consensus.cc:2932] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:24.116667 9284 raft_consensus.cc:2932] T c6956266cb444536945540ecdbaba28e P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:24.118729 7013 raft_consensus.cc:2932] T c6956266cb444536945540ecdbaba28e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:24.118904 6737 catalog_manager.cc:5484] T c6956266cb444536945540ecdbaba28e P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:24.123265 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet c6956266cb444536945540ecdbaba28e with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:24.123653 26106 tablet_replica.cc:330] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:24.123881 26106 raft_consensus.cc:2229] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:24.124379 26106 raft_consensus.cc:2258] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:24.124938 26106 ts_tablet_manager.cc:1875] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:24.127393 26106 ts_tablet_manager.cc:1888] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:24.127526 26106 log.cc:1196] T c6956266cb444536945540ecdbaba28e P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/c6956266cb444536945540ecdbaba28e
I20240426 09:50:24.128088 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet c6956266cb444536945540ecdbaba28e (table rebalancer_extra_table_38 [id=2e444a4cf06f4c748d0a8915f9dc06f7]) successfully deleted
I20240426 09:50:24.268239 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:24.271485 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:24.274171 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:24.274711 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:24.285038 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:24.285338 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:24.288014 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:24.289417 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:24.369098 9415 consensus_queue.cc:237] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:24.370373 9284 raft_consensus.cc:1261] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:24.370537 7274 raft_consensus.cc:1261] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:24.370793 25417 consensus_queue.cc:1035] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:24.370980 26029 consensus_queue.cc:1035] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:24.372952 26174 raft_consensus.cc:2932] T e64e5170d6bc4154b5ec869d535051c3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:24.373355 9284 raft_consensus.cc:2932] T e64e5170d6bc4154b5ec869d535051c3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:24.373411 7274 raft_consensus.cc:2932] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:24.376493 6731 catalog_manager.cc:5484] T e64e5170d6bc4154b5ec869d535051c3 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:24.378911 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet e64e5170d6bc4154b5ec869d535051c3 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:24.396667 26175 tablet_replica.cc:330] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:24.396831 26175 raft_consensus.cc:2229] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:24.396975 26175 raft_consensus.cc:2258] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:24.397490 26175 ts_tablet_manager.cc:1875] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:24.400913 26175 ts_tablet_manager.cc:1888] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:24.400995 26175 log.cc:1196] T e64e5170d6bc4154b5ec869d535051c3 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/e64e5170d6bc4154b5ec869d535051c3
I20240426 09:50:24.401546 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet e64e5170d6bc4154b5ec869d535051c3 (table rebalancer_extra_table_20 [id=400e31bfcc5c45e3b4efc26c62662f14]) successfully deleted
I20240426 09:50:24.519968 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:24.521200 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:24.531002 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:24.531297 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:24.532904 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:24.536609 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:24.576148 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:24.590937 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:24.653031 7011 consensus_queue.cc:237] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:24.654268 6881 raft_consensus.cc:1261] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:24.654588 9285 raft_consensus.cc:1261] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:24.654640 25640 consensus_queue.cc:1035] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:24.655221 25640 consensus_queue.cc:1035] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:24.656545 25640 raft_consensus.cc:2932] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:24.658636 9285 raft_consensus.cc:2932] T 63f066721f3441299fe5dffd07c1afed P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:24.658878 6881 raft_consensus.cc:2932] T 63f066721f3441299fe5dffd07c1afed P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:24.662072 6740 catalog_manager.cc:5484] T 63f066721f3441299fe5dffd07c1afed P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 4 to 6, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:24.667862 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 63f066721f3441299fe5dffd07c1afed with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:24.668373 26257 tablet_replica.cc:330] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:24.668512 26257 raft_consensus.cc:2229] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:24.668612 26257 raft_consensus.cc:2258] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:24.669108 26257 ts_tablet_manager.cc:1875] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:24.671770 26257 ts_tablet_manager.cc:1888] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:50:24.671854 26257 log.cc:1196] T 63f066721f3441299fe5dffd07c1afed P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/63f066721f3441299fe5dffd07c1afed
I20240426 09:50:24.672397 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 63f066721f3441299fe5dffd07c1afed (table rebalance_test_table_9 [id=ae33227b9c8a40e4af444174ba47ef9f]) successfully deleted
I20240426 09:50:24.797423 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:24.808660 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:24.812924 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:24.815625 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:24.815654 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:24.818132 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:24.820773 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:24.836922 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:24.901984 7142 consensus_queue.cc:237] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:24.903101 7274 raft_consensus.cc:1261] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:24.903174 9285 raft_consensus.cc:1261] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:24.903563 22309 consensus_queue.cc:1035] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:24.903728 25770 consensus_queue.cc:1035] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:24.905618 26328 raft_consensus.cc:2932] T 91a15bb5ed1541ca906b8603d9249690 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:24.906033 7275 raft_consensus.cc:2932] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:24.906332 9285 raft_consensus.cc:2932] T 91a15bb5ed1541ca906b8603d9249690 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:24.909634 6736 catalog_manager.cc:5484] T 91a15bb5ed1541ca906b8603d9249690 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:24.912724 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 91a15bb5ed1541ca906b8603d9249690 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:24.913594 26331 tablet_replica.cc:330] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:24.913736 26331 raft_consensus.cc:2229] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:24.913863 26331 raft_consensus.cc:2258] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:24.914363 26331 ts_tablet_manager.cc:1875] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:24.916787 26331 ts_tablet_manager.cc:1888] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:24.916858 26331 log.cc:1196] T 91a15bb5ed1541ca906b8603d9249690 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/91a15bb5ed1541ca906b8603d9249690
I20240426 09:50:24.917331 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 91a15bb5ed1541ca906b8603d9249690 (table rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]) successfully deleted
I20240426 09:50:25.051708 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:25.055014 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:25.058391 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:25.078039 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:25.083132 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:25.090454 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:25.107321 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:25.116381 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:25.181993 9546 consensus_queue.cc:237] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:25.183228 6881 raft_consensus.cc:1261] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:25.183327 7144 raft_consensus.cc:1261] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:25.183604 25583 consensus_queue.cc:1035] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:25.183792 23032 consensus_queue.cc:1035] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:25.185971 25583 raft_consensus.cc:2932] T cfe16e4114924bda8c560d4926a63923 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:25.186358 7144 raft_consensus.cc:2932] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:25.189105 6881 raft_consensus.cc:2932] T cfe16e4114924bda8c560d4926a63923 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:25.190037 6732 catalog_manager.cc:5484] T cfe16e4114924bda8c560d4926a63923 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:25.193490 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet cfe16e4114924bda8c560d4926a63923 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:25.195982 26403 tablet_replica.cc:330] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:25.196180 26403 raft_consensus.cc:2229] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:25.196348 26403 raft_consensus.cc:2258] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:25.196764 26403 ts_tablet_manager.cc:1875] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:25.199205 26403 ts_tablet_manager.cc:1888] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:25.199347 26403 log.cc:1196] T cfe16e4114924bda8c560d4926a63923 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/cfe16e4114924bda8c560d4926a63923
I20240426 09:50:25.200423 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet cfe16e4114924bda8c560d4926a63923 (table rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]) successfully deleted
I20240426 09:50:25.331776 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:25.334708 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:25.334944 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:25.343789 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:25.344424 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:25.347636 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:25.359258 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:25.399008 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:25.444065 7275 consensus_queue.cc:237] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:25.445242 6880 raft_consensus.cc:1261] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:25.445349 9417 raft_consensus.cc:1261] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:25.445606 25714 consensus_queue.cc:1035] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:25.445780 25711 consensus_queue.cc:1035] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:25.447424 25711 raft_consensus.cc:2932] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:25.447921 6880 raft_consensus.cc:2932] T d30be76596534071a3ea9a90375099f6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:25.448084 9417 raft_consensus.cc:2932] T d30be76596534071a3ea9a90375099f6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:25.451682 6732 catalog_manager.cc:5484] T d30be76596534071a3ea9a90375099f6 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:25.454676 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet d30be76596534071a3ea9a90375099f6 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:25.455209 26403 tablet_replica.cc:330] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:25.455354 26403 raft_consensus.cc:2229] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:25.455510 26403 raft_consensus.cc:2258] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:25.455984 26403 ts_tablet_manager.cc:1875] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:25.458213 26403 ts_tablet_manager.cc:1888] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:25.458274 26403 log.cc:1196] T d30be76596534071a3ea9a90375099f6 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/d30be76596534071a3ea9a90375099f6
I20240426 09:50:25.458729 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet d30be76596534071a3ea9a90375099f6 (table rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]) successfully deleted
I20240426 09:50:25.490661 9549 consensus_queue.cc:237] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:25.492005 9284 raft_consensus.cc:1261] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:25.492326 9672 raft_consensus.cc:1261] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:25.492475 9482 consensus_peers.cc:485] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b925f8cd837b4e5f8fd3f6f608d3ad31. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:25.492723 26402 consensus_queue.cc:1035] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:25.492933 26401 consensus_queue.cc:1035] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:25.495010 26402 raft_consensus.cc:2932] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:25.495472 9672 raft_consensus.cc:2932] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:25.496881 9284 raft_consensus.cc:2932] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:25.498785 6739 catalog_manager.cc:5484] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:25.521898 9672 consensus_queue.cc:237] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:25.523753 9417 raft_consensus.cc:1261] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:25.524019 9614 consensus_peers.cc:485] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a1b8658361f4c4a834e8c886909fe95. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:25.524207 25904 consensus_queue.cc:1035] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:25.524569 9284 raft_consensus.cc:1261] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:25.525017 25904 consensus_queue.cc:1035] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:25.526964 25904 raft_consensus.cc:2932] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:25.527266 9284 raft_consensus.cc:2932] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:25.529031 9417 raft_consensus.cc:2932] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:25.530922 6732 catalog_manager.cc:5484] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:25.682418 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:25.684396 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:25.690095 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:25.690526 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:25.695259 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:25.705837 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:25.707260 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:25.723304 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:25.884480 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:25.895798 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:25.899577 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:25.899577 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:25.900318 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:25.923712 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:25.924564 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:25.954667 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:25.976981 26637 ts_tablet_manager.cc:902] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:25.977674 26637 tablet_copy_client.cc:308] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:25.978271 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 6a1b8658361f4c4a834e8c886909fe95 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:48359)
I20240426 09:50:25.978375 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 6a1b8658361f4c4a834e8c886909fe95 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:48359: session id = 1aaf8befba984e199e96f53f46b50441-6a1b8658361f4c4a834e8c886909fe95
I20240426 09:50:25.979331 9685 tablet_copy_source_session.cc:205] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:25.980158 26637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1b8658361f4c4a834e8c886909fe95. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:25.982563 26637 tablet_copy_client.cc:783] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:25.982859 26637 tablet_copy_client.cc:647] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:25.999694 26637 tablet_copy_client.cc:523] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:26.001399 26637 tablet_bootstrap.cc:492] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:26.005123 26637 tablet_bootstrap.cc:492] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:26.005370 26637 tablet_bootstrap.cc:492] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:26.005519 26637 ts_tablet_manager.cc:1367] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:50:26.005955 26637 raft_consensus.cc:348] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:26.006081 26637 raft_consensus.cc:729] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:26.006405 26637 consensus_queue.cc:260] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:26.006814 26637 ts_tablet_manager.cc:1398] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240426 09:50:26.007283 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-6a1b8658361f4c4a834e8c886909fe95 received from {username='slave'} at 127.0.99.131:48359
I20240426 09:50:26.007359 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-6a1b8658361f4c4a834e8c886909fe95 on tablet 6a1b8658361f4c4a834e8c886909fe95 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:26.066454 26699 ts_tablet_manager.cc:902] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:26.076206 26699 tablet_copy_client.cc:308] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:50:26.076706 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet b925f8cd837b4e5f8fd3f6f608d3ad31 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:39323)
I20240426 09:50:26.076812 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet b925f8cd837b4e5f8fd3f6f608d3ad31 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:39323: session id = 3b41d87e7bfb419f8aa7500100129b6c-b925f8cd837b4e5f8fd3f6f608d3ad31
I20240426 09:50:26.077834 9569 tablet_copy_source_session.cc:205] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:26.080070 26699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b925f8cd837b4e5f8fd3f6f608d3ad31. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:26.085538 26699 tablet_copy_client.cc:783] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:26.085816 26699 tablet_copy_client.cc:647] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:26.087309 26699 tablet_copy_client.cc:523] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:26.088603 26699 tablet_bootstrap.cc:492] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:26.091857 26699 tablet_bootstrap.cc:492] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:26.092114 26699 tablet_bootstrap.cc:492] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:26.092258 26699 ts_tablet_manager.cc:1367] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240426 09:50:26.092628 26699 raft_consensus.cc:348] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:26.092720 26699 raft_consensus.cc:729] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:26.092828 26699 consensus_queue.cc:260] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:26.094352 26699 ts_tablet_manager.cc:1398] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:26.094827 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-b925f8cd837b4e5f8fd3f6f608d3ad31 received from {username='slave'} at 127.0.99.134:39323
I20240426 09:50:26.094936 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-b925f8cd837b4e5f8fd3f6f608d3ad31 on tablet b925f8cd837b4e5f8fd3f6f608d3ad31 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:26.172535 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:26.172647 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:26.175592 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:26.178947 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:26.182901 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:26.184984 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:26.186535 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:26.203574 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:26.384047 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:26.386328 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:26.388018 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:26.392318 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:26.395701 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:26.403566 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:26.425827 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:26.439271 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:26.566073 7143 raft_consensus.cc:1203] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:26.627152 9415 raft_consensus.cc:1203] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:26.804661 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:26.805466 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:26.815692 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:26.825314 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:26.826030 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:26.827922 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:26.828747 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:26.834923 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:26.914966 9673 consensus_queue.cc:237] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:26.916414 7144 raft_consensus.cc:1261] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:26.916422 9415 raft_consensus.cc:1261] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:26.916793 26532 consensus_queue.cc:1035] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:26.916923 26529 consensus_queue.cc:1035] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:26.918848 26841 raft_consensus.cc:2932] T 6a1b8658361f4c4a834e8c886909fe95 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:26.919255 7144 raft_consensus.cc:2932] T 6a1b8658361f4c4a834e8c886909fe95 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:26.919342 9415 raft_consensus.cc:2932] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:26.922688 6734 catalog_manager.cc:5484] T 6a1b8658361f4c4a834e8c886909fe95 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:26.926049 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1b8658361f4c4a834e8c886909fe95 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:26.927650 26851 tablet_replica.cc:330] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:26.927875 26851 raft_consensus.cc:2229] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:26.928023 26851 raft_consensus.cc:2258] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:26.928436 26851 ts_tablet_manager.cc:1875] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:26.931351 26851 ts_tablet_manager.cc:1888] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:26.931432 26851 log.cc:1196] T 6a1b8658361f4c4a834e8c886909fe95 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/6a1b8658361f4c4a834e8c886909fe95
I20240426 09:50:26.931994 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 6a1b8658361f4c4a834e8c886909fe95 (table rebalancer_extra_table_26 [id=cf104cc998fb42728cf09391eacd2375]) successfully deleted
I20240426 09:50:27.068940 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:27.069350 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:27.073035 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:27.073529 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:27.099249 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:27.106204 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:27.106920 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:27.117897 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:27.168067 9546 consensus_queue.cc:237] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:27.169262 9415 raft_consensus.cc:1261] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:27.169628 9673 raft_consensus.cc:1261] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:27.169615 23032 consensus_queue.cc:1035] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:27.170826 23032 consensus_queue.cc:1035] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:27.172452 26402 raft_consensus.cc:2932] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:27.174896 9417 raft_consensus.cc:2932] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:27.177305 9673 raft_consensus.cc:2932] T b925f8cd837b4e5f8fd3f6f608d3ad31 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:27.177898 6734 catalog_manager.cc:5484] T b925f8cd837b4e5f8fd3f6f608d3ad31 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:27.181847 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet b925f8cd837b4e5f8fd3f6f608d3ad31 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:27.182169 26851 tablet_replica.cc:330] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:27.182327 26851 raft_consensus.cc:2229] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:27.182489 26851 raft_consensus.cc:2258] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:27.182986 26851 ts_tablet_manager.cc:1875] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:27.184998 26851 ts_tablet_manager.cc:1888] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:27.185067 26851 log.cc:1196] T b925f8cd837b4e5f8fd3f6f608d3ad31 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/b925f8cd837b4e5f8fd3f6f608d3ad31
I20240426 09:50:27.185796 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet b925f8cd837b4e5f8fd3f6f608d3ad31 (table rebalancer_extra_table_46 [id=55e575f9bda845118699d93860a7b3a4]) successfully deleted
I20240426 09:50:27.632503 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:27.648660 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:27.658190 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:27.658564 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:27.662997 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:27.674270 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:27.686447 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:27.697925 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:27.824405 7274 consensus_queue.cc:237] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:27.825809 6881 raft_consensus.cc:1261] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:27.825871 7011 raft_consensus.cc:1261] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:27.826267 27003 consensus_queue.cc:1035] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:27.826520 26420 consensus_queue.cc:1035] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:27.826689 7208 consensus_peers.cc:485] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 61bc946b265243198d1cb16247c43b7f. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:27.828554 26420 raft_consensus.cc:2932] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:27.829044 7011 raft_consensus.cc:2932] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:27.830608 6881 raft_consensus.cc:2932] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:27.832149 6735 catalog_manager.cc:5484] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:27.855815 9284 consensus_queue.cc:237] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:27.857602 9417 raft_consensus.cc:1261] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:27.857689 9673 raft_consensus.cc:1261] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:27.858021 27025 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:50:27.858017 9219 consensus_peers.cc:485] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6cb4534d05c49ed859b4abe61097ac8. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:27.858318 26718 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:27.860051 27025 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:27.860661 9673 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:27.863768 6739 catalog_manager.cc:5484] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:27.864701 9417 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:27.891011 7143 consensus_queue.cc:237] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:27.892446 7274 raft_consensus.cc:1261] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:27.892582 9284 raft_consensus.cc:1261] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:27.892966 26328 consensus_queue.cc:1035] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:27.893177 26919 consensus_queue.cc:1035] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:27.894445 7077 consensus_peers.cc:485] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7eecd02c620e4b1eb17548a94e6a9ac1. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:27.895602 26328 raft_consensus.cc:2932] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:27.896063 7274 raft_consensus.cc:2932] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:27.897621 9284 raft_consensus.cc:2932] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:27.899621 6733 catalog_manager.cc:5484] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:27.923121 9673 consensus_queue.cc:237] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:27.924554 9284 raft_consensus.cc:1261] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:27.924890 9546 raft_consensus.cc:1261] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:27.925099 27064 consensus_queue.cc:1035] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:27.925310 9614 consensus_peers.cc:485] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4613f57999fa4a6baf5ccfd102f5c73b. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:27.925493 26842 consensus_queue.cc:1035] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:27.927206 27064 raft_consensus.cc:2932] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:27.927618 9284 raft_consensus.cc:2932] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:27.930208 9546 raft_consensus.cc:2932] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:27.931087 6733 catalog_manager.cc:5484] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:27.958537 9546 consensus_queue.cc:237] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:27.959908 9673 raft_consensus.cc:1261] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:27.959966 9284 raft_consensus.cc:1261] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:27.960136 9484 consensus_peers.cc:485] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6981747bd8c44d2bb56f3d5bb56dbb1. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:27.960285 23032 consensus_queue.cc:1035] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:27.960603 23032 consensus_queue.cc:1035] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:27.962291 26986 raft_consensus.cc:2932] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:27.962819 9284 raft_consensus.cc:2932] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:27.965485 9673 raft_consensus.cc:2932] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:27.966338 6735 catalog_manager.cc:5484] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:27.989641 9546 consensus_queue.cc:237] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:27.991050 6881 raft_consensus.cc:1261] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:27.991079 9417 raft_consensus.cc:1261] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:27.991312 9484 consensus_peers.cc:485] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01d9726ca8b641178824a7fa5977bbdf. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:27.991487 23032 consensus_queue.cc:1035] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:27.991664 26402 consensus_queue.cc:1035] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:27.993818 23032 raft_consensus.cc:2932] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:27.995672 9417 raft_consensus.cc:2932] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:27.996018 6881 raft_consensus.cc:2932] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:27.999800 6733 catalog_manager.cc:5484] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:28.024108 7011 consensus_queue.cc:237] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:28.025573 7275 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.025564 9546 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:28.025733 6946 consensus_peers.cc:485] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7799bbe0befb41d0a155ff51f41c23ad. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.026034 26705 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.026216 25187 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.028221 26705 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:28.028527 9546 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:28.032042 6735 catalog_manager.cc:5484] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:28.033465 7275 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:28.053745 6881 consensus_queue.cc:237] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:28.055553 7011 raft_consensus.cc:1261] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.055660 7143 raft_consensus.cc:1261] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.056114 26984 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.056301 24896 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.058296 24896 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.058732 7143 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
W20240426 09:50:28.061383 6816 consensus_peers.cc:485] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1cec7dfa2ba1466d9d5fc6e55baf280b. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.062281 6738 catalog_manager.cc:5484] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.066426 7011 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.085044 6881 consensus_queue.cc:237] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:28.086411 9417 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:28.086561 9284 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20240426 09:50:28.086632 6815 consensus_peers.cc:485] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2770cd77ebb642e3a52183a3871e33fa. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.086934 26984 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:28.087112 24896 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:28.088826 26984 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:28.089257 9284 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:28.090780 9417 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:28.092649 6735 catalog_manager.cc:5484] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 4 to 5, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 2 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:28.120383 7144 consensus_queue.cc:237] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:28.122072 9673 raft_consensus.cc:1261] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.122093 6881 raft_consensus.cc:1261] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:28.122231 7077 consensus_peers.cc:485] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 845d1058274a41d49b6ac67571cb5439. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.122414 26328 consensus_queue.cc:1035] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.122583 26919 consensus_queue.cc:1035] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.124384 26328 raft_consensus.cc:2932] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:28.124790 6881 raft_consensus.cc:2932] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:28.126328 9673 raft_consensus.cc:2932] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:28.127780 6733 catalog_manager.cc:5484] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:28.155580 9284 consensus_queue.cc:237] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:28.157219 9546 raft_consensus.cc:1261] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:28.157299 9221 consensus_peers.cc:485] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6983bc4d17d44f998a749f7ec95bb305. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.157532 9415 raft_consensus.cc:1261] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.157857 27028 consensus_queue.cc:1035] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.158107 26983 consensus_queue.cc:1035] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.160225 27028 raft_consensus.cc:2932] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:28.160598 9415 raft_consensus.cc:2932] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:28.164930 6735 catalog_manager.cc:5484] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:28.171162 9546 raft_consensus.cc:2932] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:28.197343 9672 consensus_queue.cc:237] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:28.199445 7143 raft_consensus.cc:1261] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.199445 7275 raft_consensus.cc:1261] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:28.200181 9615 consensus_peers.cc:485] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6f6f0978441e4ce69c6e40805e495fd0. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.200218 26529 consensus_queue.cc:1035] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.200497 27067 consensus_queue.cc:1035] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.202972 26529 raft_consensus.cc:2932] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.203439 7275 raft_consensus.cc:2932] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.206063 7144 raft_consensus.cc:2932] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.206689 6733 catalog_manager.cc:5484] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.228312 9415 consensus_queue.cc:237] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:28.229899 7275 raft_consensus.cc:1261] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.229905 7011 raft_consensus.cc:1261] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:28.230224 9350 consensus_peers.cc:485] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b66a5a6cf3348ffbaa56679985e9aa2. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.230352 27230 consensus_queue.cc:1035] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.230566 26918 consensus_queue.cc:1035] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.232398 26918 raft_consensus.cc:2932] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:28.234489 7011 raft_consensus.cc:2932] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:28.235284 7275 raft_consensus.cc:2932] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:28.236140 6733 catalog_manager.cc:5484] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:28.257964 9415 consensus_queue.cc:237] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:28.259752 7011 raft_consensus.cc:1261] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:28.260262 9350 consensus_peers.cc:485] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b08039d1ab5e4b108ae21a24c507c876. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.260282 7275 raft_consensus.cc:1261] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.260329 27230 consensus_queue.cc:1035] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.260833 27230 consensus_queue.cc:1035] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.262667 27230 raft_consensus.cc:2932] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:28.264811 7275 raft_consensus.cc:2932] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:28.266044 7011 raft_consensus.cc:2932] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:28.267557 6733 catalog_manager.cc:5484] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:28.285993 7011 consensus_queue.cc:237] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:28.287647 7144 raft_consensus.cc:1261] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.287628 9546 raft_consensus.cc:1261] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.288123 25187 consensus_queue.cc:1035] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.288311 26705 consensus_queue.cc:1035] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:28.288482 6948 consensus_peers.cc:485] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 348e90f5c63e4eda89b87fc103896302. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.290266 26705 raft_consensus.cc:2932] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:28.291139 7144 raft_consensus.cc:2932] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:28.292222 9546 raft_consensus.cc:2932] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:28.295200 6738 catalog_manager.cc:5484] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:28.297937 27274 ts_tablet_manager.cc:902] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:28.300679 27274 tablet_copy_client.cc:308] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:28.301134 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 7eecd02c620e4b1eb17548a94e6a9ac1 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:49245)
I20240426 09:50:28.301256 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 7eecd02c620e4b1eb17548a94e6a9ac1 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:49245: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-7eecd02c620e4b1eb17548a94e6a9ac1
I20240426 09:50:28.302326 7164 tablet_copy_source_session.cc:205] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.303296 27274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eecd02c620e4b1eb17548a94e6a9ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.305806 27274 tablet_copy_client.cc:783] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.305989 27274 tablet_copy_client.cc:647] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.308363 27274 tablet_copy_client.cc:523] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.309720 27274 tablet_bootstrap.cc:492] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:50:28.313370 27274 tablet_bootstrap.cc:492] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.313623 27274 tablet_bootstrap.cc:492] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:50:28.313786 27274 ts_tablet_manager.cc:1367] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:50:28.314602 27274 raft_consensus.cc:348] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:28.314738 27274 raft_consensus.cc:729] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.314903 27274 consensus_queue.cc:260] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:28.315295 27274 ts_tablet_manager.cc:1398] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240426 09:50:28.315801 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-7eecd02c620e4b1eb17548a94e6a9ac1 received from {username='slave'} at 127.0.99.135:49245
I20240426 09:50:28.315896 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-7eecd02c620e4b1eb17548a94e6a9ac1 on tablet 7eecd02c620e4b1eb17548a94e6a9ac1 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:50:28.317529 27291 ts_tablet_manager.cc:902] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:28.318066 27291 tablet_copy_client.cc:308] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:28.319617 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet e6cb4534d05c49ed859b4abe61097ac8 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:36823)
I20240426 09:50:28.319715 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet e6cb4534d05c49ed859b4abe61097ac8 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:36823: session id = 1aaf8befba984e199e96f53f46b50441-e6cb4534d05c49ed859b4abe61097ac8
I20240426 09:50:28.320617 9306 tablet_copy_source_session.cc:205] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.321511 9672 consensus_queue.cc:237] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:28.322170 27291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6cb4534d05c49ed859b4abe61097ac8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.323208 9284 raft_consensus.cc:1261] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:28.323789 26529 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:28.323840 9546 raft_consensus.cc:1261] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:28.324177 9614 consensus_peers.cc:485] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.324420 27067 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:28.324824 27291 tablet_copy_client.cc:783] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.324982 27291 tablet_copy_client.cc:647] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.326301 26529 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:28.326696 27291 tablet_copy_client.cc:523] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.326691 9546 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:28.328214 27291 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:28.329836 9284 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:28.330090 6733 catalog_manager.cc:5484] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:28.336603 27291 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.336898 27291 tablet_bootstrap.cc:492] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:28.337085 27291 ts_tablet_manager.cc:1367] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20240426 09:50:28.337574 27291 raft_consensus.cc:348] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:28.337720 27291 raft_consensus.cc:729] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.337849 27291 consensus_queue.cc:260] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:28.338116 27291 ts_tablet_manager.cc:1398] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:28.339440 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-e6cb4534d05c49ed859b4abe61097ac8 received from {username='slave'} at 127.0.99.131:36823
I20240426 09:50:28.339547 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-e6cb4534d05c49ed859b4abe61097ac8 on tablet e6cb4534d05c49ed859b4abe61097ac8 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:28.355381 7144 consensus_queue.cc:237] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:28.356945 7013 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:28.357061 7077 consensus_peers.cc:485] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 838bef0a35c046e39b465f2ea743350f. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.357349 25770 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:28.358448 9546 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:28.358851 25770 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:28.360483 27047 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:28.362076 9546 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:28.362216 7013 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:28.364058 6735 catalog_manager.cc:5484] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:28.376575 27328 ts_tablet_manager.cc:902] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:28.377043 27330 ts_tablet_manager.cc:902] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:50:28.377097 27328 tablet_copy_client.cc:308] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:50:28.377517 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet 01d9726ca8b641178824a7fa5977bbdf from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:59655)
I20240426 09:50:28.377558 27330 tablet_copy_client.cc:308] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:50:28.377629 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet 01d9726ca8b641178824a7fa5977bbdf from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:59655: session id = e3394a4be9ea4def8aa39b333c82c5ff-01d9726ca8b641178824a7fa5977bbdf
I20240426 09:50:28.377962 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 61bc946b265243198d1cb16247c43b7f from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:41659)
I20240426 09:50:28.378041 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 61bc946b265243198d1cb16247c43b7f from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:41659: session id = 3b41d87e7bfb419f8aa7500100129b6c-61bc946b265243198d1cb16247c43b7f
I20240426 09:50:28.378407 9569 tablet_copy_source_session.cc:205] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.378962 7295 tablet_copy_source_session.cc:205] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.379238 27328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d9726ca8b641178824a7fa5977bbdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.381513 27328 tablet_copy_client.cc:783] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.381657 27328 tablet_copy_client.cc:647] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.382309 27330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61bc946b265243198d1cb16247c43b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.384652 27330 tablet_copy_client.cc:783] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.384804 27330 tablet_copy_client.cc:647] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.386237 27328 tablet_copy_client.cc:523] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.387674 27328 tablet_bootstrap.cc:492] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:28.389994 27330 tablet_copy_client.cc:523] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.391364 27330 tablet_bootstrap.cc:492] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:28.394469 27328 tablet_bootstrap.cc:492] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.394716 27328 tablet_bootstrap.cc:492] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:28.394882 27328 ts_tablet_manager.cc:1367] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20240426 09:50:28.395367 27328 raft_consensus.cc:348] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:28.395519 27328 raft_consensus.cc:729] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.395655 27328 consensus_queue.cc:260] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:28.395968 27328 ts_tablet_manager.cc:1398] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:28.396977 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-01d9726ca8b641178824a7fa5977bbdf received from {username='slave'} at 127.0.99.132:59655
I20240426 09:50:28.397051 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-01d9726ca8b641178824a7fa5977bbdf on tablet 01d9726ca8b641178824a7fa5977bbdf with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:28.397294 27330 tablet_bootstrap.cc:492] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.397541 27330 tablet_bootstrap.cc:492] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:28.397665 27330 ts_tablet_manager.cc:1367] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20240426 09:50:28.398067 27330 raft_consensus.cc:348] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:28.398160 27330 raft_consensus.cc:729] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.398293 27330 consensus_queue.cc:260] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:28.398568 27330 ts_tablet_manager.cc:1398] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:28.399961 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-61bc946b265243198d1cb16247c43b7f received from {username='slave'} at 127.0.99.134:41659
I20240426 09:50:28.400049 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-61bc946b265243198d1cb16247c43b7f on tablet 61bc946b265243198d1cb16247c43b7f with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:28.407497 6881 consensus_queue.cc:237] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:28.409035 7275 raft_consensus.cc:1261] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:28.409091 6816 consensus_peers.cc:485] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e78db4ae5cd4c40a3ed0478f8e9dee6. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:28.409431 24896 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.409490 9415 raft_consensus.cc:1261] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:28.411015 24896 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:28.411448 27346 ts_tablet_manager.cc:902] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: Initiating tablet copy from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:28.412124 27346 tablet_copy_client.cc:308] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.135:36979
I20240426 09:50:28.412631 9569 tablet_copy_service.cc:140] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Received BeginTabletCopySession request for tablet d6981747bd8c44d2bb56f3d5bb56dbb1 from peer c244e02f3cf847c0a5b69e1059f1071c ({username='slave'} at 127.0.99.129:35867)
I20240426 09:50:28.412734 9569 tablet_copy_service.cc:161] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Beginning new tablet copy session on tablet d6981747bd8c44d2bb56f3d5bb56dbb1 from peer c244e02f3cf847c0a5b69e1059f1071c at {username='slave'} at 127.0.99.129:35867: session id = c244e02f3cf847c0a5b69e1059f1071c-d6981747bd8c44d2bb56f3d5bb56dbb1
I20240426 09:50:28.413688 9569 tablet_copy_source_session.cc:205] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.413834 24896 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.415366 9415 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.416172 7275 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.417551 27346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6981747bd8c44d2bb56f3d5bb56dbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.417510 6733 catalog_manager.cc:5484] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index -1 to 2, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:28.420640 27346 tablet_copy_client.cc:783] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.421065 27346 tablet_copy_client.cc:647] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.425024 27346 tablet_copy_client.cc:523] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.426491 27346 tablet_bootstrap.cc:492] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap starting.
I20240426 09:50:28.430234 27346 tablet_bootstrap.cc:492] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.430490 27346 tablet_bootstrap.cc:492] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: Bootstrap complete.
I20240426 09:50:28.430648 27346 ts_tablet_manager.cc:1367] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.001s
I20240426 09:50:28.431113 27346 raft_consensus.cc:348] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:28.431253 27346 raft_consensus.cc:729] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.431380 27346 consensus_queue.cc:260] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:28.431703 27346 ts_tablet_manager.cc:1398] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:28.432802 9569 tablet_copy_service.cc:342] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Request end of tablet copy session c244e02f3cf847c0a5b69e1059f1071c-d6981747bd8c44d2bb56f3d5bb56dbb1 received from {username='slave'} at 127.0.99.129:35867
I20240426 09:50:28.432884 9569 tablet_copy_service.cc:433] P b8e863bc0edf4940bbf3e9f9b2bbb3e5: ending tablet copy session c244e02f3cf847c0a5b69e1059f1071c-d6981747bd8c44d2bb56f3d5bb56dbb1 on tablet d6981747bd8c44d2bb56f3d5bb56dbb1 with peer c244e02f3cf847c0a5b69e1059f1071c
I20240426 09:50:28.443526 27366 ts_tablet_manager.cc:902] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:28.444098 27366 tablet_copy_client.cc:308] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:28.444586 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 4613f57999fa4a6baf5ccfd102f5c73b from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:59473)
I20240426 09:50:28.444681 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 4613f57999fa4a6baf5ccfd102f5c73b from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:59473: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-4613f57999fa4a6baf5ccfd102f5c73b
I20240426 09:50:28.445401 9685 tablet_copy_source_session.cc:205] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.446223 27366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4613f57999fa4a6baf5ccfd102f5c73b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.448431 27366 tablet_copy_client.cc:783] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.448592 27366 tablet_copy_client.cc:647] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.449607 27366 tablet_copy_client.cc:523] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.450685 27366 tablet_bootstrap.cc:492] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:28.453850 27366 tablet_bootstrap.cc:492] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.454077 27366 tablet_bootstrap.cc:492] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:28.454221 27366 ts_tablet_manager.cc:1367] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240426 09:50:28.454881 27366 raft_consensus.cc:348] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:28.454993 27366 raft_consensus.cc:729] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.455129 27366 consensus_queue.cc:260] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:28.455420 27366 ts_tablet_manager.cc:1398] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:28.456630 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-4613f57999fa4a6baf5ccfd102f5c73b received from {username='slave'} at 127.0.99.130:59473
I20240426 09:50:28.456710 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-4613f57999fa4a6baf5ccfd102f5c73b on tablet 4613f57999fa4a6baf5ccfd102f5c73b with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:28.465030 27291 ts_tablet_manager.cc:902] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:28.465497 27291 tablet_copy_client.cc:308] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:28.465955 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 7799bbe0befb41d0a155ff51f41c23ad from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:54895)
I20240426 09:50:28.466038 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 7799bbe0befb41d0a155ff51f41c23ad from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:54895: session id = 1aaf8befba984e199e96f53f46b50441-7799bbe0befb41d0a155ff51f41c23ad
I20240426 09:50:28.467128 7033 tablet_copy_source_session.cc:205] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.467985 27291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7799bbe0befb41d0a155ff51f41c23ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.470245 27291 tablet_copy_client.cc:783] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.470393 27291 tablet_copy_client.cc:647] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.477411 27291 tablet_copy_client.cc:523] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.482860 27291 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:28.487910 27291 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.488201 27291 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:28.488373 27291 ts_tablet_manager.cc:1367] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240426 09:50:28.488828 27291 raft_consensus.cc:348] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:28.488991 27291 raft_consensus.cc:729] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.489372 27291 consensus_queue.cc:260] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:28.490602 27291 ts_tablet_manager.cc:1398] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:28.491087 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-7799bbe0befb41d0a155ff51f41c23ad received from {username='slave'} at 127.0.99.131:54895
I20240426 09:50:28.491178 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-7799bbe0befb41d0a155ff51f41c23ad on tablet 7799bbe0befb41d0a155ff51f41c23ad with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:28.534937 27407 ts_tablet_manager.cc:902] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:28.536226 27407 tablet_copy_client.cc:308] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:28.536728 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:54839)
I20240426 09:50:28.536841 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 1cec7dfa2ba1466d9d5fc6e55baf280b from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:54839: session id = a692602e1a2e4794877e6fb92a7b9962-1cec7dfa2ba1466d9d5fc6e55baf280b
I20240426 09:50:28.537900 6902 tablet_copy_source_session.cc:205] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.539088 27407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.542285 27407 tablet_copy_client.cc:783] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.542539 27407 tablet_copy_client.cc:647] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.547919 27407 tablet_copy_client.cc:523] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.549793 27407 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:28.556030 27407 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.556937 27407 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:28.557876 27407 ts_tablet_manager.cc:1367] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20240426 09:50:28.558447 27407 raft_consensus.cc:348] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:28.558646 27407 raft_consensus.cc:729] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.558842 27407 consensus_queue.cc:260] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:28.560525 27407 ts_tablet_manager.cc:1398] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:50:28.561286 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-1cec7dfa2ba1466d9d5fc6e55baf280b received from {username='slave'} at 127.0.99.133:54839
I20240426 09:50:28.561374 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-1cec7dfa2ba1466d9d5fc6e55baf280b on tablet 1cec7dfa2ba1466d9d5fc6e55baf280b with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:28.567409 27330 ts_tablet_manager.cc:902] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:28.567893 27330 tablet_copy_client.cc:308] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:28.568689 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 845d1058274a41d49b6ac67571cb5439 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:52457)
I20240426 09:50:28.568812 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 845d1058274a41d49b6ac67571cb5439 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:52457: session id = 3b41d87e7bfb419f8aa7500100129b6c-845d1058274a41d49b6ac67571cb5439
I20240426 09:50:28.569897 7164 tablet_copy_source_session.cc:205] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.570951 27330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 845d1058274a41d49b6ac67571cb5439. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.573395 27330 tablet_copy_client.cc:783] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.573546 27330 tablet_copy_client.cc:647] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.575261 27366 ts_tablet_manager.cc:902] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:28.575727 27366 tablet_copy_client.cc:308] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:28.576584 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 2770cd77ebb642e3a52183a3871e33fa from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:34321)
I20240426 09:50:28.576699 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 2770cd77ebb642e3a52183a3871e33fa from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:34321: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-2770cd77ebb642e3a52183a3871e33fa
I20240426 09:50:28.577587 6902 tablet_copy_source_session.cc:205] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.578723 27330 tablet_copy_client.cc:523] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.579593 27366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2770cd77ebb642e3a52183a3871e33fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.580168 27330 tablet_bootstrap.cc:492] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:28.581820 27366 tablet_copy_client.cc:783] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.585116 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:28.583765 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:28.581982 27366 tablet_copy_client.cc:647] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.587867 27330 tablet_bootstrap.cc:492] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.588182 27330 tablet_bootstrap.cc:492] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:28.588344 27330 ts_tablet_manager.cc:1367] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20240426 09:50:28.589016 27330 raft_consensus.cc:348] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:28.589135 27330 raft_consensus.cc:729] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.591063 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:28.591125 27366 tablet_copy_client.cc:523] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.591958 27330 consensus_queue.cc:260] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:28.592654 27366 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:28.596707 27330 ts_tablet_manager.cc:1398] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:50:28.596854 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:28.599390 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-845d1058274a41d49b6ac67571cb5439 received from {username='slave'} at 127.0.99.134:52457
I20240426 09:50:28.599478 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-845d1058274a41d49b6ac67571cb5439 on tablet 845d1058274a41d49b6ac67571cb5439 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:28.606098 27328 ts_tablet_manager.cc:902] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:28.606583 27328 tablet_copy_client.cc:308] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:28.611310 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet 6983bc4d17d44f998a749f7ec95bb305 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:38007)
I20240426 09:50:28.611423 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet 6983bc4d17d44f998a749f7ec95bb305 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:38007: session id = e3394a4be9ea4def8aa39b333c82c5ff-6983bc4d17d44f998a749f7ec95bb305
I20240426 09:50:28.612371 9306 tablet_copy_source_session.cc:205] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.614327 27328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6983bc4d17d44f998a749f7ec95bb305. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.619424 27328 tablet_copy_client.cc:783] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.619603 27328 tablet_copy_client.cc:647] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.623426 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:28.639870 27274 ts_tablet_manager.cc:902] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:28.640385 27274 tablet_copy_client.cc:308] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:28.643352 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet b08039d1ab5e4b108ae21a24c507c876 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:35103)
I20240426 09:50:28.643496 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet b08039d1ab5e4b108ae21a24c507c876 from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:35103: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-b08039d1ab5e4b108ae21a24c507c876
I20240426 09:50:28.644572 9437 tablet_copy_source_session.cc:205] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.644693 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:28.645630 27274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b08039d1ab5e4b108ae21a24c507c876. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.648051 27274 tablet_copy_client.cc:783] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.648221 27274 tablet_copy_client.cc:647] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.649327 27274 tablet_copy_client.cc:523] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.650031 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:28.650310 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:28.651715 27274 tablet_bootstrap.cc:492] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:50:28.676383 27366 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.676694 27366 tablet_bootstrap.cc:492] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:28.676864 27366 ts_tablet_manager.cc:1367] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.084s user 0.005s sys 0.000s
I20240426 09:50:28.677337 27366 raft_consensus.cc:348] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:28.677453 27366 raft_consensus.cc:729] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.678462 27366 consensus_queue.cc:260] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:28.678932 27366 ts_tablet_manager.cc:1398] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:28.679584 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-2770cd77ebb642e3a52183a3871e33fa received from {username='slave'} at 127.0.99.130:34321
I20240426 09:50:28.679669 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-2770cd77ebb642e3a52183a3871e33fa on tablet 2770cd77ebb642e3a52183a3871e33fa with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:28.686069 27328 tablet_copy_client.cc:523] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.687705 27328 tablet_bootstrap.cc:492] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:28.688539 27274 tablet_bootstrap.cc:492] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.688795 27274 tablet_bootstrap.cc:492] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:50:28.688987 27274 ts_tablet_manager.cc:1367] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.002s
I20240426 09:50:28.689358 27274 raft_consensus.cc:348] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:28.689456 27274 raft_consensus.cc:729] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.689602 27274 consensus_queue.cc:260] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:28.689900 27274 ts_tablet_manager.cc:1398] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20240426 09:50:28.691887 27328 tablet_bootstrap.cc:492] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.692157 27328 tablet_bootstrap.cc:492] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:28.692337 27328 ts_tablet_manager.cc:1367] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240426 09:50:28.692793 27328 raft_consensus.cc:348] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:28.692934 27328 raft_consensus.cc:729] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.693059 27328 consensus_queue.cc:260] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:28.693315 27328 ts_tablet_manager.cc:1398] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:28.693673 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-b08039d1ab5e4b108ae21a24c507c876 received from {username='slave'} at 127.0.99.135:35103
I20240426 09:50:28.693755 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-b08039d1ab5e4b108ae21a24c507c876 on tablet b08039d1ab5e4b108ae21a24c507c876 with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:50:28.696943 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-6983bc4d17d44f998a749f7ec95bb305 received from {username='slave'} at 127.0.99.132:38007
I20240426 09:50:28.697042 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-6983bc4d17d44f998a749f7ec95bb305 on tablet 6983bc4d17d44f998a749f7ec95bb305 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:28.711678 27328 ts_tablet_manager.cc:902] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:28.712112 27328 tablet_copy_client.cc:308] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:28.712556 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 348e90f5c63e4eda89b87fc103896302 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:32839)
I20240426 09:50:28.712671 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 348e90f5c63e4eda89b87fc103896302 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:32839: session id = e3394a4be9ea4def8aa39b333c82c5ff-348e90f5c63e4eda89b87fc103896302
I20240426 09:50:28.713699 7033 tablet_copy_source_session.cc:205] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.714555 27328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 348e90f5c63e4eda89b87fc103896302. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.716552 27328 tablet_copy_client.cc:783] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.716702 27328 tablet_copy_client.cc:647] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.717746 27328 tablet_copy_client.cc:523] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.718878 27328 tablet_bootstrap.cc:492] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:28.723493 27328 tablet_bootstrap.cc:492] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.723798 27328 tablet_bootstrap.cc:492] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:28.723963 27328 ts_tablet_manager.cc:1367] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20240426 09:50:28.724434 27328 raft_consensus.cc:348] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:28.724571 27328 raft_consensus.cc:729] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.724704 27328 consensus_queue.cc:260] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:28.725085 27328 ts_tablet_manager.cc:1398] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:28.725636 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-348e90f5c63e4eda89b87fc103896302 received from {username='slave'} at 127.0.99.132:32839
I20240426 09:50:28.725718 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-348e90f5c63e4eda89b87fc103896302 on tablet 348e90f5c63e4eda89b87fc103896302 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:28.753903 7275 raft_consensus.cc:1203] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:28.760690 27472 ts_tablet_manager.cc:902] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:28.761371 27472 tablet_copy_client.cc:308] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:28.762048 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet 5b66a5a6cf3348ffbaa56679985e9aa2 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:44543)
I20240426 09:50:28.762154 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet 5b66a5a6cf3348ffbaa56679985e9aa2 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:44543: session id = 558c4ecd0ece460d9591aa7a3c799d19-5b66a5a6cf3348ffbaa56679985e9aa2
I20240426 09:50:28.763116 9437 tablet_copy_source_session.cc:205] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.764012 27472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b66a5a6cf3348ffbaa56679985e9aa2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.767213 27472 tablet_copy_client.cc:783] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.767436 27472 tablet_copy_client.cc:647] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.769060 27472 tablet_copy_client.cc:523] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.770500 27472 tablet_bootstrap.cc:492] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:50:28.774428 27472 tablet_bootstrap.cc:492] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.774693 27472 tablet_bootstrap.cc:492] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:50:28.774884 27472 ts_tablet_manager.cc:1367] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:50:28.775389 27472 raft_consensus.cc:348] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:28.775496 27472 raft_consensus.cc:729] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.775628 27472 consensus_queue.cc:260] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:28.776037 27472 ts_tablet_manager.cc:1398] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:28.777020 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-5b66a5a6cf3348ffbaa56679985e9aa2 received from {username='slave'} at 127.0.99.136:44543
I20240426 09:50:28.777102 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-5b66a5a6cf3348ffbaa56679985e9aa2 on tablet 5b66a5a6cf3348ffbaa56679985e9aa2 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:50:28.778414 27407 ts_tablet_manager.cc:902] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:28.778970 27407 tablet_copy_client.cc:308] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:28.781651 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 6f6f0978441e4ce69c6e40805e495fd0 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:33703)
I20240426 09:50:28.781749 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 6f6f0978441e4ce69c6e40805e495fd0 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:33703: session id = a692602e1a2e4794877e6fb92a7b9962-6f6f0978441e4ce69c6e40805e495fd0
I20240426 09:50:28.782727 9685 tablet_copy_source_session.cc:205] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.784409 27407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f6f0978441e4ce69c6e40805e495fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.787704 27407 tablet_copy_client.cc:783] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.787886 27407 tablet_copy_client.cc:647] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.793985 27407 tablet_copy_client.cc:523] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.795574 27407 tablet_bootstrap.cc:492] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:28.800220 27407 tablet_bootstrap.cc:492] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.800791 27407 tablet_bootstrap.cc:492] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:28.801008 27407 ts_tablet_manager.cc:1367] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20240426 09:50:28.805894 27407 raft_consensus.cc:348] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:28.806143 27407 raft_consensus.cc:729] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.806339 27407 consensus_queue.cc:260] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:28.807792 27407 ts_tablet_manager.cc:1398] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:50:28.808174 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-6f6f0978441e4ce69c6e40805e495fd0 received from {username='slave'} at 127.0.99.133:33703
I20240426 09:50:28.808251 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-6f6f0978441e4ce69c6e40805e495fd0 on tablet 6f6f0978441e4ce69c6e40805e495fd0 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:28.821364 7144 raft_consensus.cc:1203] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:28.884150 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:28.890131 27407 ts_tablet_manager.cc:902] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:28.890614 27407 tablet_copy_client.cc:308] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:28.891059 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 from peer a692602e1a2e4794877e6fb92a7b9962 ({username='slave'} at 127.0.99.133:54839)
I20240426 09:50:28.891168 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 from peer a692602e1a2e4794877e6fb92a7b9962 at {username='slave'} at 127.0.99.133:54839: session id = a692602e1a2e4794877e6fb92a7b9962-2e78db4ae5cd4c40a3ed0478f8e9dee6
I20240426 09:50:28.892027 6902 tablet_copy_source_session.cc:205] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.902698 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:28.904960 9549 raft_consensus.cc:1203] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:28.905254 27330 ts_tablet_manager.cc:902] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:28.905694 27330 tablet_copy_client.cc:308] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:28.906204 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 6112d2390c994fefb74b8e7f75d57d17 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:46185)
I20240426 09:50:28.906301 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 6112d2390c994fefb74b8e7f75d57d17 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:46185: session id = 3b41d87e7bfb419f8aa7500100129b6c-6112d2390c994fefb74b8e7f75d57d17
I20240426 09:50:28.907346 9685 tablet_copy_source_session.cc:205] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.910665 27330 ts_tablet_manager.cc:1875] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240426 09:50:28.913172 27330 ts_tablet_manager.cc:1888] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20240426 09:50:28.913347 27330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6112d2390c994fefb74b8e7f75d57d17. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.916831 6880 raft_consensus.cc:1203] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:28.917968 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:28.918514 27472 ts_tablet_manager.cc:902] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:28.918931 27472 tablet_copy_client.cc:308] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:28.922191 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:28.922281 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:28.922863 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet 838bef0a35c046e39b465f2ea743350f from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:32985)
I20240426 09:50:28.922974 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet 838bef0a35c046e39b465f2ea743350f from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:32985: session id = 558c4ecd0ece460d9591aa7a3c799d19-838bef0a35c046e39b465f2ea743350f
I20240426 09:50:28.923211 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:28.923934 7164 tablet_copy_source_session.cc:205] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:28.924391 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:28.927474 27407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.934165 27472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 838bef0a35c046e39b465f2ea743350f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:28.936444 27330 tablet_copy_client.cc:783] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.936566 27472 tablet_copy_client.cc:783] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.936617 27330 tablet_copy_client.cc:647] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.936730 27472 tablet_copy_client.cc:647] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.937918 27472 tablet_copy_client.cc:523] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.939270 27472 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:50:28.939867 27407 tablet_copy_client.cc:783] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:28.940057 27407 tablet_copy_client.cc:647] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:28.941071 27407 tablet_copy_client.cc:523] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.942380 27407 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap starting.
I20240426 09:50:28.947190 27407 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.947427 27407 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: Bootstrap complete.
I20240426 09:50:28.947568 27407 ts_tablet_manager.cc:1367] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:50:28.948004 27407 raft_consensus.cc:348] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:28.948107 27407 raft_consensus.cc:729] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.948216 27407 consensus_queue.cc:260] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:28.948503 27407 ts_tablet_manager.cc:1398] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:28.949385 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session a692602e1a2e4794877e6fb92a7b9962-2e78db4ae5cd4c40a3ed0478f8e9dee6 received from {username='slave'} at 127.0.99.133:54839
I20240426 09:50:28.949466 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session a692602e1a2e4794877e6fb92a7b9962-2e78db4ae5cd4c40a3ed0478f8e9dee6 on tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 with peer a692602e1a2e4794877e6fb92a7b9962
I20240426 09:50:28.960673 27330 tablet_copy_client.cc:523] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:28.962551 27330 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:28.973280 27330 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.973549 27330 tablet_bootstrap.cc:492] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:28.973711 27330 ts_tablet_manager.cc:1367] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20240426 09:50:28.976461 27472 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:28.976742 27472 tablet_bootstrap.cc:492] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:50:28.976897 27472 ts_tablet_manager.cc:1367] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.038s user 0.005s sys 0.000s
I20240426 09:50:28.977357 27472 raft_consensus.cc:348] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:28.977501 27472 raft_consensus.cc:729] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.977619 27472 consensus_queue.cc:260] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:28.978579 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:28.983881 27330 raft_consensus.cc:348] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:28.984020 27330 raft_consensus.cc:729] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:28.987479 27330 consensus_queue.cc:260] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:28.987823 27330 ts_tablet_manager.cc:1398] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
I20240426 09:50:28.988281 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-6112d2390c994fefb74b8e7f75d57d17 received from {username='slave'} at 127.0.99.134:46185
I20240426 09:50:28.988351 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-6112d2390c994fefb74b8e7f75d57d17 on tablet 6112d2390c994fefb74b8e7f75d57d17 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:28.991493 27472 ts_tablet_manager.cc:1398] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.015s user 0.001s sys 0.000s
I20240426 09:50:28.992910 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-838bef0a35c046e39b465f2ea743350f received from {username='slave'} at 127.0.99.136:32985
I20240426 09:50:28.992995 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-838bef0a35c046e39b465f2ea743350f on tablet 838bef0a35c046e39b465f2ea743350f with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:50:28.993466 9415 raft_consensus.cc:1203] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.003944 9286 raft_consensus.cc:1203] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.020429 7275 raft_consensus.cc:1203] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.027539 7143 raft_consensus.cc:1203] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.029762 7011 raft_consensus.cc:1203] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.083550 7011 raft_consensus.cc:1203] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.4->[2.5-2.5] Dedup: 2.5->[]
I20240426 09:50:29.087442 9546 raft_consensus.cc:1203] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.168519 9415 raft_consensus.cc:1203] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.208194 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:29.208851 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:29.214342 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:29.227648 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:29.227912 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:29.240482 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:29.240525 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:29.244252 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:29.244585 7275 raft_consensus.cc:1203] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.281975 9673 raft_consensus.cc:1203] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.338265 9286 raft_consensus.cc:1203] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.339257 9415 raft_consensus.cc:1203] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:29.417251 9673 raft_consensus.cc:1203] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:29.445163 9286 raft_consensus.cc:1203] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:29.452236 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:29.452629 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:29.453711 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:29.467674 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:29.479727 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:29.493562 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:29.520433 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:29.535686 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:29.716560 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:29.718746 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:29.724318 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:29.725425 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:29.733387 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:29.734616 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:29.750967 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:29.761773 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:29.832824 9415 consensus_queue.cc:237] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:29.834053 9549 raft_consensus.cc:1261] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:29.834250 7274 raft_consensus.cc:1261] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:29.834466 27233 consensus_queue.cc:1035] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:29.834698 27502 consensus_queue.cc:1035] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:29.837505 27502 raft_consensus.cc:2932] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:29.837827 7274 raft_consensus.cc:2932] T b08039d1ab5e4b108ae21a24c507c876 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:29.840663 9549 raft_consensus.cc:2932] T b08039d1ab5e4b108ae21a24c507c876 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:29.841584 6732 catalog_manager.cc:5484] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:29.846904 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet b08039d1ab5e4b108ae21a24c507c876 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47654
I20240426 09:50:29.847295 27716 tablet_replica.cc:330] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:50:29.847543 27716 raft_consensus.cc:2229] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:29.847779 27716 raft_consensus.cc:2258] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:29.848311 27716 ts_tablet_manager.cc:1875] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:29.850689 27716 ts_tablet_manager.cc:1888] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:29.850852 27716 log.cc:1196] T b08039d1ab5e4b108ae21a24c507c876 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/b08039d1ab5e4b108ae21a24c507c876
I20240426 09:50:29.851513 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet b08039d1ab5e4b108ae21a24c507c876 (table rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]) successfully deleted
I20240426 09:50:30.000319 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:30.001036 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:30.016960 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:30.017079 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:30.018939 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:30.031617 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:30.032455 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:30.056843 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:30.118149 9673 consensus_queue.cc:237] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:30.119499 9285 raft_consensus.cc:1261] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:30.119499 7275 raft_consensus.cc:1261] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:30.119935 27067 consensus_queue.cc:1035] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:30.120317 27067 consensus_queue.cc:1035] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:30.122169 27067 raft_consensus.cc:2932] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:30.122610 7275 raft_consensus.cc:2932] T 6f6f0978441e4ce69c6e40805e495fd0 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:30.126785 6740 catalog_manager.cc:5484] T 6f6f0978441e4ce69c6e40805e495fd0 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:30.128350 9285 raft_consensus.cc:2932] T 6f6f0978441e4ce69c6e40805e495fd0 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:30.131206 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f6f0978441e4ce69c6e40805e495fd0 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:30.131569 27788 tablet_replica.cc:330] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:30.131784 27788 raft_consensus.cc:2229] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:30.131932 27788 raft_consensus.cc:2258] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:30.132329 27788 ts_tablet_manager.cc:1875] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:30.134491 27788 ts_tablet_manager.cc:1888] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:30.134563 27788 log.cc:1196] T 6f6f0978441e4ce69c6e40805e495fd0 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/6f6f0978441e4ce69c6e40805e495fd0
I20240426 09:50:30.135197 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 6f6f0978441e4ce69c6e40805e495fd0 (table rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]) successfully deleted
I20240426 09:50:30.274369 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:30.284484 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:30.285840 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:30.286923 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:30.286928 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:30.298888 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:30.307539 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:30.317631 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:30.374593 7143 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "838bef0a35c046e39b465f2ea743350f"
dest_uuid: "1aaf8befba984e199e96f53f46b50441"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:38264
I20240426 09:50:30.374727 7143 raft_consensus.cc:595] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:30.513681 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:30.515584 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:30.517648 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:30.519670 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:30.527448 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:30.530535 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:30.530853 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:30.534931 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:30.635497 25770 raft_consensus.cc:979] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: : Instructing follower 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:50:30.635627 25770 raft_consensus.cc:1067] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Signalling peer 0a0bfe28bcc0435ebe9a54a9d6416899 to start an election
I20240426 09:50:30.636054 7011 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "838bef0a35c046e39b465f2ea743350f"
dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
from {username='slave'} at 127.0.99.131:54895
I20240426 09:50:30.636222 7011 raft_consensus.cc:482] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:30.636283 7011 raft_consensus.cc:3037] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:30.637091 7011 raft_consensus.cc:504] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:30.637454 7011 leader_election.cc:288] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Requested vote from peers 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:30.637773 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838bef0a35c046e39b465f2ea743350f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:50:30.637897 7143 raft_consensus.cc:3032] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:30.637945 7143 raft_consensus.cc:729] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:50:30.638093 7143 consensus_queue.cc:260] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:30.638108 9546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838bef0a35c046e39b465f2ea743350f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:50:30.638135 9669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "838bef0a35c046e39b465f2ea743350f" candidate_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:50:30.638234 9546 raft_consensus.cc:3037] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:30.638286 7143 raft_consensus.cc:3037] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:30.638269 9669 raft_consensus.cc:3037] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:30.639350 7143 raft_consensus.cc:2454] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:50:30.639359 9546 raft_consensus.cc:2454] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:50:30.639839 6946 leader_election.cc:302] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:50:30.639844 9669 raft_consensus.cc:2454] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a0bfe28bcc0435ebe9a54a9d6416899 in term 2.
I20240426 09:50:30.640053 26985 raft_consensus.cc:2781] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:30.640132 26985 raft_consensus.cc:686] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Becoming Leader. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:50:30.640262 26985 consensus_queue.cc:237] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:30.643558 6733 catalog_manager.cc:5484] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: term changed from 1 to 2, leader changed from 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) to 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130). New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:30.753135 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:30.757432 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:30.760135 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:30.762666 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:30.770983 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:30.776701 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:30.802053 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:30.834398 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:30.883306 9285 consensus_queue.cc:237] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:30.884723 9415 raft_consensus.cc:1261] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:30.884734 7275 raft_consensus.cc:1261] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:30.885100 27947 consensus_queue.cc:1035] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:30.885291 27025 consensus_queue.cc:1035] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:30.887545 27025 raft_consensus.cc:2932] T 6983bc4d17d44f998a749f7ec95bb305 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:30.887853 9415 raft_consensus.cc:2932] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:30.888350 7275 raft_consensus.cc:2932] T 6983bc4d17d44f998a749f7ec95bb305 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:30.891502 6734 catalog_manager.cc:5484] T 6983bc4d17d44f998a749f7ec95bb305 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:30.895609 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 6983bc4d17d44f998a749f7ec95bb305 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:30.898205 27966 tablet_replica.cc:330] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:30.898449 27966 raft_consensus.cc:2229] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:30.898638 27966 raft_consensus.cc:2258] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:30.899217 27966 ts_tablet_manager.cc:1875] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:30.902021 27966 ts_tablet_manager.cc:1888] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:30.902197 27966 log.cc:1196] T 6983bc4d17d44f998a749f7ec95bb305 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/6983bc4d17d44f998a749f7ec95bb305
I20240426 09:50:30.902894 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 6983bc4d17d44f998a749f7ec95bb305 (table rebalancer_extra_table_18 [id=b4db4d0283b640499c229b753e54b5bf]) successfully deleted
I20240426 09:50:31.026600 7144 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:31.027915 27516 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:31.053941 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.057883 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:31.062945 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:31.064718 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:31.068050 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.080358 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.098788 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:31.107295 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:31.156761 9669 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:31.157150 27516 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:31.161708 9546 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:31.163437 26985 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:31.174857 7013 consensus_queue.cc:237] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:31.176282 7275 raft_consensus.cc:1261] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:31.176647 9546 raft_consensus.cc:1261] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:31.176723 26985 consensus_queue.cc:1035] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:31.177242 26985 consensus_queue.cc:1035] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:31.178370 27516 raft_consensus.cc:2932] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:31.178941 7275 raft_consensus.cc:2932] T 348e90f5c63e4eda89b87fc103896302 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:31.182736 6738 catalog_manager.cc:5484] T 348e90f5c63e4eda89b87fc103896302 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:31.185695 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 348e90f5c63e4eda89b87fc103896302 with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:31.187095 28044 tablet_replica.cc:330] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:31.187240 28044 raft_consensus.cc:2229] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:31.187402 28044 raft_consensus.cc:2258] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:31.187955 28044 ts_tablet_manager.cc:1875] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:31.190415 28044 ts_tablet_manager.cc:1888] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:31.190579 28044 log.cc:1196] T 348e90f5c63e4eda89b87fc103896302 P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/348e90f5c63e4eda89b87fc103896302
I20240426 09:50:31.191251 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 348e90f5c63e4eda89b87fc103896302 (table rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]) successfully deleted
I20240426 09:50:31.192742 9546 raft_consensus.cc:2932] T 348e90f5c63e4eda89b87fc103896302 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:31.320443 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:31.320973 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:31.321236 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:31.322105 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.330953 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.334308 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:31.344597 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:31.355397 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.422902 7144 consensus_queue.cc:237] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:31.424455 9415 raft_consensus.cc:1261] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:31.424942 6880 raft_consensus.cc:1261] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:31.425082 25770 consensus_queue.cc:1035] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:31.425364 27825 consensus_queue.cc:1035] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:31.427139 27825 raft_consensus.cc:2932] T 845d1058274a41d49b6ac67571cb5439 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:31.427687 9415 raft_consensus.cc:2932] T 845d1058274a41d49b6ac67571cb5439 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:31.427731 6880 raft_consensus.cc:2932] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:31.431344 6740 catalog_manager.cc:5484] T 845d1058274a41d49b6ac67571cb5439 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:31.435319 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 845d1058274a41d49b6ac67571cb5439 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:31.435678 28121 tablet_replica.cc:330] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:31.435904 28121 raft_consensus.cc:2229] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:31.436051 28121 raft_consensus.cc:2258] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:31.436534 28121 ts_tablet_manager.cc:1875] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:31.439417 28121 ts_tablet_manager.cc:1888] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:31.439481 28121 log.cc:1196] T 845d1058274a41d49b6ac67571cb5439 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/845d1058274a41d49b6ac67571cb5439
I20240426 09:50:31.440564 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 845d1058274a41d49b6ac67571cb5439 (table rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]) successfully deleted
I20240426 09:50:31.570657 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.574334 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.580535 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:31.583273 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:31.602210 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:31.603420 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:31.604547 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:31.614776 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.684259 6880 consensus_queue.cc:237] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:31.685792 9285 raft_consensus.cc:1261] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:31.685773 7144 raft_consensus.cc:1261] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:31.686133 27764 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:31.686332 26382 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:31.688488 27764 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:31.689026 7144 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:31.692085 6733 catalog_manager.cc:5484] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:31.693013 9285 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:31.696463 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47654
I20240426 09:50:31.697081 28192 tablet_replica.cc:330] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:50:31.697245 28192 raft_consensus.cc:2229] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:31.697491 28192 raft_consensus.cc:2258] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:31.698025 28192 ts_tablet_manager.cc:1875] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:31.700639 28192 ts_tablet_manager.cc:1888] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:31.700726 28192 log.cc:1196] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/1cec7dfa2ba1466d9d5fc6e55baf280b
I20240426 09:50:31.701848 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 1cec7dfa2ba1466d9d5fc6e55baf280b (table rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]) successfully deleted
I20240426 09:50:31.839749 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.850600 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:31.852090 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.852650 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:31.859622 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:31.861264 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:31.875195 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:31.882939 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:31.968339 7013 consensus_queue.cc:237] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:31.969723 7275 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:31.970345 7144 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:31.970723 27516 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:31.970952 26985 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:31.973385 26985 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:31.973799 7274 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:31.975495 7144 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:31.979640 6732 catalog_manager.cc:5484] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:31.987915 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet 7799bbe0befb41d0a155ff51f41c23ad with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:31.988314 28270 tablet_replica.cc:330] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:31.988469 28270 raft_consensus.cc:2229] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:31.988597 28270 raft_consensus.cc:2258] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:31.988989 28270 ts_tablet_manager.cc:1875] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:31.993086 28270 ts_tablet_manager.cc:1888] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:31.993161 28270 log.cc:1196] T 7799bbe0befb41d0a155ff51f41c23ad P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/7799bbe0befb41d0a155ff51f41c23ad
I20240426 09:50:31.993640 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet 7799bbe0befb41d0a155ff51f41c23ad (table rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]) successfully deleted
I20240426 09:50:32.138988 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:32.140810 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.147953 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:32.149205 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.151906 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:32.152184 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:32.178892 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.187719 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:32.251809 9549 consensus_queue.cc:237] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:32.253185 6880 raft_consensus.cc:1261] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:32.253206 7275 raft_consensus.cc:1261] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:32.253584 27687 consensus_queue.cc:1035] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:32.253787 27935 consensus_queue.cc:1035] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:32.255652 28331 raft_consensus.cc:2932] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:32.256028 7275 raft_consensus.cc:2932] T 01d9726ca8b641178824a7fa5977bbdf P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:32.256047 6880 raft_consensus.cc:2932] T 01d9726ca8b641178824a7fa5977bbdf P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:32.259383 6732 catalog_manager.cc:5484] T 01d9726ca8b641178824a7fa5977bbdf P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:32.263454 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d9726ca8b641178824a7fa5977bbdf with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:32.264029 28336 tablet_replica.cc:330] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:32.264192 28336 raft_consensus.cc:2229] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:32.264418 28336 raft_consensus.cc:2258] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:32.264952 28336 ts_tablet_manager.cc:1875] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:32.267297 28336 ts_tablet_manager.cc:1888] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:32.267426 28336 log.cc:1196] T 01d9726ca8b641178824a7fa5977bbdf P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/01d9726ca8b641178824a7fa5977bbdf
I20240426 09:50:32.268024 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 01d9726ca8b641178824a7fa5977bbdf (table rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]) successfully deleted
I20240426 09:50:32.398023 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:32.402086 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:32.402228 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:32.402923 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.410897 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:32.422832 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.449781 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:32.450446 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.505260 9549 consensus_queue.cc:237] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } }
I20240426 09:50:32.506395 9284 raft_consensus.cc:1261] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:32.506453 6880 raft_consensus.cc:1261] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer b8e863bc0edf4940bbf3e9f9b2bbb3e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:32.506824 28331 consensus_queue.cc:1035] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:32.506984 27687 consensus_queue.cc:1035] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:32.508783 27687 raft_consensus.cc:2932] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:32.509145 9284 raft_consensus.cc:2932] T d6981747bd8c44d2bb56f3d5bb56dbb1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:32.509267 6880 raft_consensus.cc:2932] T d6981747bd8c44d2bb56f3d5bb56dbb1 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:32.512888 6738 catalog_manager.cc:5484] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } }
I20240426 09:50:32.515973 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet d6981747bd8c44d2bb56f3d5bb56dbb1 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:32.516323 28411 tablet_replica.cc:330] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:32.516568 28411 raft_consensus.cc:2229] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:32.516949 28411 raft_consensus.cc:2258] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:32.517557 28411 ts_tablet_manager.cc:1875] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:32.519678 28411 ts_tablet_manager.cc:1888] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:32.519788 28411 log.cc:1196] T d6981747bd8c44d2bb56f3d5bb56dbb1 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/d6981747bd8c44d2bb56f3d5bb56dbb1
I20240426 09:50:32.520403 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet d6981747bd8c44d2bb56f3d5bb56dbb1 (table rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]) successfully deleted
I20240426 09:50:32.673417 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:32.674898 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:32.677542 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.680001 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.684237 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:32.686527 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:32.703747 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.709915 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:32.784466 9673 consensus_queue.cc:237] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:32.787618 7011 raft_consensus.cc:1261] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:32.787645 9549 raft_consensus.cc:1261] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:32.788345 27067 consensus_queue.cc:1035] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:32.788563 26842 consensus_queue.cc:1035] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:32.790634 28479 raft_consensus.cc:2932] T 4613f57999fa4a6baf5ccfd102f5c73b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:32.790735 7011 raft_consensus.cc:2932] T 4613f57999fa4a6baf5ccfd102f5c73b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:32.790805 9549 raft_consensus.cc:2932] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:32.794539 6739 catalog_manager.cc:5484] T 4613f57999fa4a6baf5ccfd102f5c73b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:32.797791 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 4613f57999fa4a6baf5ccfd102f5c73b with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:32.800117 28482 tablet_replica.cc:330] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:32.801374 28482 raft_consensus.cc:2229] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:32.801525 28482 raft_consensus.cc:2258] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:32.801942 28482 ts_tablet_manager.cc:1875] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:32.806043 28482 ts_tablet_manager.cc:1888] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:32.806113 28482 log.cc:1196] T 4613f57999fa4a6baf5ccfd102f5c73b P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/4613f57999fa4a6baf5ccfd102f5c73b
I20240426 09:50:32.806731 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 4613f57999fa4a6baf5ccfd102f5c73b (table rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]) successfully deleted
I20240426 09:50:32.945087 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:32.949849 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:32.956794 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.956966 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:32.958387 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:32.958971 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.963388 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:32.979657 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:33.053107 6880 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa"
dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:59844
I20240426 09:50:33.053264 6880 raft_consensus.cc:595] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Received request to transfer leadership
I20240426 09:50:33.176795 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:33.179949 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.181643 6862 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.194489 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:33.199411 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:33.199492 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:33.203575 27764 raft_consensus.cc:979] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: : Instructing follower 3b41d87e7bfb419f8aa7500100129b6c to start an election
I20240426 09:50:33.203661 27764 raft_consensus.cc:1067] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Signalling peer 3b41d87e7bfb419f8aa7500100129b6c to start an election
I20240426 09:50:33.204080 9417 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa"
dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
from {username='slave'} at 127.0.99.129:59303
I20240426 09:50:33.204223 9417 raft_consensus.cc:482] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:33.204274 9417 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:50:33.205325 9417 raft_consensus.cc:504] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:33.206441 9285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:50:33.206560 9285 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:50:33.207551 9285 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 3.
I20240426 09:50:33.207828 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.217613 6880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c244e02f3cf847c0a5b69e1059f1071c"
I20240426 09:50:33.217741 6880 raft_consensus.cc:3032] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Stepping down as leader of term 2
I20240426 09:50:33.217790 6880 raft_consensus.cc:729] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c244e02f3cf847c0a5b69e1059f1071c, State: Running, Role: LEADER
I20240426 09:50:33.217921 6880 consensus_queue.cc:260] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:33.218091 6880 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:50:33.219066 6880 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 3.
I20240426 09:50:33.219549 9352 leader_election.cc:302] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3b41d87e7bfb419f8aa7500100129b6c, a692602e1a2e4794877e6fb92a7b9962, c244e02f3cf847c0a5b69e1059f1071c; no voters:
I20240426 09:50:33.219751 27706 raft_consensus.cc:2781] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 3 FOLLOWER]: Leader election won for term 3
I20240426 09:50:33.219919 27706 raft_consensus.cc:686] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 3 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:50:33.220081 27706 consensus_queue.cc:237] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:33.223239 6731 catalog_manager.cc:5484] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 2 to 3, leader changed from c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 3 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:33.229176 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2770cd77ebb642e3a52183a3871e33fa" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:50:33.229308 7011 raft_consensus.cc:3037] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:50:33.229735 9417 leader_election.cc:288] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 3 election: Requested vote from peers c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:33.235973 7011 raft_consensus.cc:2454] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 3.
I20240426 09:50:33.241386 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:33.437073 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:33.437511 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:33.439997 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:33.445685 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.445775 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:33.445960 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.448614 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.461093 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:33.533306 9284 consensus_queue.cc:237] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:33.534497 7144 raft_consensus.cc:1261] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:33.534860 9417 raft_consensus.cc:1261] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:33.534894 28406 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:33.535248 28406 consensus_queue.cc:1035] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:33.536582 27025 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:33.536978 7144 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:33.537361 9417 raft_consensus.cc:2932] T e6cb4534d05c49ed859b4abe61097ac8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:33.541113 6738 catalog_manager.cc:5484] T e6cb4534d05c49ed859b4abe61097ac8 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 4 to 5, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:33.543936 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet e6cb4534d05c49ed859b4abe61097ac8 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:33.544607 28670 tablet_replica.cc:330] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:33.544950 28670 raft_consensus.cc:2229] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:33.545342 28670 raft_consensus.cc:2258] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:33.545881 28670 ts_tablet_manager.cc:1875] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:33.548283 28670 ts_tablet_manager.cc:1888] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:33.548446 28670 log.cc:1196] T e6cb4534d05c49ed859b4abe61097ac8 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/e6cb4534d05c49ed859b4abe61097ac8
I20240426 09:50:33.549131 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet e6cb4534d05c49ed859b4abe61097ac8 (table rebalancer_extra_table_40 [id=adbd88ce011c44009649284af6387cee]) successfully deleted
I20240426 09:50:33.614128 6880 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 3 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20240426 09:50:33.614607 28587 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:33.684912 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:33.688602 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:33.688454 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:33.701402 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.705461 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.715317 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:33.715368 9284 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 3 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20240426 09:50:33.718991 28587 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:33.728513 7013 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 3 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20240426 09:50:33.728926 27706 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:33.743949 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:33.756716 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.822553 7144 consensus_queue.cc:237] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:33.823819 9549 raft_consensus.cc:1261] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:33.823925 9285 raft_consensus.cc:1261] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:33.824272 25770 consensus_queue.cc:1035] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:33.824445 27825 consensus_queue.cc:1035] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:33.826385 28751 raft_consensus.cc:2932] T 7eecd02c620e4b1eb17548a94e6a9ac1 P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:33.827138 9285 raft_consensus.cc:2932] T 7eecd02c620e4b1eb17548a94e6a9ac1 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:33.827237 9549 raft_consensus.cc:2932] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:33.831039 6734 catalog_manager.cc:5484] T 7eecd02c620e4b1eb17548a94e6a9ac1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:33.834069 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eecd02c620e4b1eb17548a94e6a9ac1 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:33.834471 28756 tablet_replica.cc:330] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:33.834678 28756 raft_consensus.cc:2229] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:33.834861 28756 raft_consensus.cc:2258] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:33.835354 28756 ts_tablet_manager.cc:1875] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:33.837301 28756 ts_tablet_manager.cc:1888] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:33.837452 28756 log.cc:1196] T 7eecd02c620e4b1eb17548a94e6a9ac1 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/7eecd02c620e4b1eb17548a94e6a9ac1
I20240426 09:50:33.838449 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 7eecd02c620e4b1eb17548a94e6a9ac1 (table rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]) successfully deleted
I20240426 09:50:33.983939 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:33.987444 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:33.992038 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:34.003288 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:34.007016 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:34.023672 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:34.036636 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:34.036837 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:34.105317 7274 consensus_queue.cc:237] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:34.106585 9417 raft_consensus.cc:1261] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:34.106616 6882 raft_consensus.cc:1261] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:34.107028 28640 consensus_queue.cc:1035] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:34.107205 28386 consensus_queue.cc:1035] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:34.109073 28823 raft_consensus.cc:2932] T 61bc946b265243198d1cb16247c43b7f P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:34.109402 9417 raft_consensus.cc:2932] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:34.109447 6882 raft_consensus.cc:2932] T 61bc946b265243198d1cb16247c43b7f P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:34.112680 6732 catalog_manager.cc:5484] T 61bc946b265243198d1cb16247c43b7f P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:34.116648 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet 61bc946b265243198d1cb16247c43b7f with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47654
I20240426 09:50:34.117555 28824 tablet_replica.cc:330] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:50:34.117982 28824 raft_consensus.cc:2229] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:34.118268 28824 raft_consensus.cc:2258] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:34.119314 28824 ts_tablet_manager.cc:1875] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:34.122367 28824 ts_tablet_manager.cc:1888] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:34.122438 28824 log.cc:1196] T 61bc946b265243198d1cb16247c43b7f P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/61bc946b265243198d1cb16247c43b7f
I20240426 09:50:34.123127 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 61bc946b265243198d1cb16247c43b7f (table rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]) successfully deleted
I20240426 09:50:34.462029 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:34.464668 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:34.473870 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:34.482712 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:34.484992 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:34.489333 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:34.499765 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:34.503788 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:34.562160 6882 consensus_queue.cc:237] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } }
I20240426 09:50:34.563455 9284 raft_consensus.cc:1261] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:34.563522 9415 raft_consensus.cc:1261] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:34.564067 28402 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:34.565177 28402 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:34.566793 28402 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:34.568725 9284 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:34.571576 9415 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:34.572502 6733 catalog_manager.cc:5484] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } }
I20240426 09:50:34.579885 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:34.580252 28918 tablet_replica.cc:330] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:34.580403 28918 raft_consensus.cc:2229] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:34.580547 28918 raft_consensus.cc:2258] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:34.580986 28918 ts_tablet_manager.cc:1875] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:34.583645 28918 ts_tablet_manager.cc:1888] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:34.583724 28918 log.cc:1196] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/2e78db4ae5cd4c40a3ed0478f8e9dee6
I20240426 09:50:34.584328 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 (table rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]) successfully deleted
I20240426 09:50:34.715147 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:34.719048 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:34.725627 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:34.725971 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:34.727406 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:34.733250 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:34.757951 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:34.766264 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:34.830957 9673 consensus_queue.cc:237] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:34.832175 9415 raft_consensus.cc:1261] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:34.832245 9549 raft_consensus.cc:1261] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:34.832615 26842 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:34.832851 28732 consensus_queue.cc:1035] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:34.835146 26842 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:34.835618 9415 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:34.839015 6734 catalog_manager.cc:5484] T 6112d2390c994fefb74b8e7f75d57d17 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 4 to 5, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:34.840618 9549 raft_consensus.cc:2932] T 6112d2390c994fefb74b8e7f75d57d17 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:34.843055 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 6112d2390c994fefb74b8e7f75d57d17 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:34.843884 28987 tablet_replica.cc:330] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:34.844089 28987 raft_consensus.cc:2229] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:34.844254 28987 raft_consensus.cc:2258] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:34.844923 28987 ts_tablet_manager.cc:1875] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:34.847945 28987 ts_tablet_manager.cc:1888] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:34.848017 28987 log.cc:1196] T 6112d2390c994fefb74b8e7f75d57d17 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/6112d2390c994fefb74b8e7f75d57d17
I20240426 09:50:34.848655 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 6112d2390c994fefb74b8e7f75d57d17 (table rebalancer_extra_table_28 [id=e352a9b35deb4880ad95a5974209406e]) successfully deleted
I20240426 09:50:35.014998 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:35.022718 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.025475 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:35.027119 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:35.028302 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:35.028618 9660 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240426 09:50:35.042557 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.083932 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:35.119385 9415 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5b66a5a6cf3348ffbaa56679985e9aa2"
dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:39958
I20240426 09:50:35.119534 9415 raft_consensus.cc:595] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:35.197136 28977 raft_consensus.cc:979] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: : Instructing follower 558c4ecd0ece460d9591aa7a3c799d19 to start an election
I20240426 09:50:35.197264 28977 raft_consensus.cc:1067] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Signalling peer 558c4ecd0ece460d9591aa7a3c799d19 to start an election
I20240426 09:50:35.197984 9673 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5b66a5a6cf3348ffbaa56679985e9aa2"
dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
from {username='slave'} at 127.0.99.134:46185
I20240426 09:50:35.198136 9673 raft_consensus.cc:482] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:35.198226 9673 raft_consensus.cc:3037] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:35.199380 9673 raft_consensus.cc:504] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:35.199844 9673 leader_election.cc:288] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 2 election: Requested vote from peers 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203), 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:50:35.200450 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b66a5a6cf3348ffbaa56679985e9aa2" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:50:35.200587 7274 raft_consensus.cc:3037] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:35.201735 7274 raft_consensus.cc:2454] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 2.
I20240426 09:50:35.203675 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b66a5a6cf3348ffbaa56679985e9aa2" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:50:35.203804 7011 raft_consensus.cc:3037] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:35.203989 9415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b66a5a6cf3348ffbaa56679985e9aa2" candidate_uuid: "558c4ecd0ece460d9591aa7a3c799d19" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
I20240426 09:50:35.204133 9415 raft_consensus.cc:3032] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:35.204188 9415 raft_consensus.cc:729] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:50:35.204339 9415 consensus_queue.cc:260] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:35.204567 9415 raft_consensus.cc:3037] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:35.204973 7011 raft_consensus.cc:2454] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 2.
I20240426 09:50:35.205627 9614 leader_election.cc:302] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:50:35.205664 9415 raft_consensus.cc:2454] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 558c4ecd0ece460d9591aa7a3c799d19 in term 2.
I20240426 09:50:35.206323 26842 raft_consensus.cc:2781] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:35.206421 26842 raft_consensus.cc:686] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 LEADER]: Becoming Leader. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Running, Role: LEADER
I20240426 09:50:35.206568 26842 consensus_queue.cc:237] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:35.210328 6733 catalog_manager.cc:5484] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: term changed from 1 to 2, leader changed from 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) to 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136). New cstate: current_term: 2 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:35.259349 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:35.265493 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:35.266156 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:35.268464 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.275434 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:35.275662 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.278159 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.333604 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:35.505405 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.505405 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:35.505765 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.525918 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:35.530908 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:35.539793 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.540585 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:35.540892 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:35.592314 7011 raft_consensus.cc:1261] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:35.592778 28732 consensus_queue.cc:1035] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:35.619683 7013 consensus_queue.cc:237] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:35.620854 9673 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:35.621089 9548 raft_consensus.cc:1261] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:35.621215 27516 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:35.621590 27516 consensus_queue.cc:1035] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:35.623224 27516 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:35.627380 9669 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:35.627887 6731 catalog_manager.cc:5484] T 838bef0a35c046e39b465f2ea743350f P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 4 to 6, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 2 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:35.631426 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 838bef0a35c046e39b465f2ea743350f with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:35.631758 29181 tablet_replica.cc:330] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:35.631911 29181 raft_consensus.cc:2229] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:35.632050 29181 raft_consensus.cc:2258] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:35.632558 29181 ts_tablet_manager.cc:1875] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:35.634088 9548 raft_consensus.cc:2932] T 838bef0a35c046e39b465f2ea743350f P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:35.635466 29181 ts_tablet_manager.cc:1888] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:50:35.635545 29181 log.cc:1196] T 838bef0a35c046e39b465f2ea743350f P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/838bef0a35c046e39b465f2ea743350f
I20240426 09:50:35.636211 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 838bef0a35c046e39b465f2ea743350f (table rebalancer_extra_table_24 [id=f68275209e754a138a0f29033601c75a]) successfully deleted
I20240426 09:50:35.701511 9417 raft_consensus.cc:1261] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:35.701938 26842 consensus_queue.cc:1035] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:35.706274 7274 raft_consensus.cc:1261] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240426 09:50:35.706693 28732 consensus_queue.cc:1035] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:35.831193 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.838882 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.844148 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:35.846098 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:35.848030 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:35.848838 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:35.850598 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:35.879459 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:35.946899 9417 consensus_queue.cc:237] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:35.948199 7013 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 3 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240426 09:50:35.948207 9285 raft_consensus.cc:1261] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 3 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240426 09:50:35.948812 29294 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:35.949488 28977 consensus_queue.cc:1035] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240426 09:50:35.950811 28726 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c [term 3 LEADER]: Committing config change with OpId 3.7: config changed from index 5 to 7, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:35.951243 9285 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 [term 3 FOLLOWER]: Committing config change with OpId 3.7: config changed from index 5 to 7, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:35.953439 7013 raft_consensus.cc:2932] T 2770cd77ebb642e3a52183a3871e33fa P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 3 FOLLOWER]: Committing config change with OpId 3.7: config changed from index 5 to 7, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:35.955591 6732 catalog_manager.cc:5484] T 2770cd77ebb642e3a52183a3871e33fa P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 5 to 7, VOTER c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129) evicted. New cstate: current_term: 3 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:35.960213 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet 2770cd77ebb642e3a52183a3871e33fa with delete_type TABLET_DATA_TOMBSTONED (TS c244e02f3cf847c0a5b69e1059f1071c not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:44196
I20240426 09:50:35.961547 29297 tablet_replica.cc:330] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: stopping tablet replica
I20240426 09:50:35.961750 29297 raft_consensus.cc:2229] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 3 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:35.961898 29297 raft_consensus.cc:2258] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c [term 3 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:35.962358 29297 ts_tablet_manager.cc:1875] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:35.964707 29297 ts_tablet_manager.cc:1888] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.6
I20240426 09:50:35.964787 29297 log.cc:1196] T 2770cd77ebb642e3a52183a3871e33fa P c244e02f3cf847c0a5b69e1059f1071c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/wals/2770cd77ebb642e3a52183a3871e33fa
I20240426 09:50:35.965371 6726 catalog_manager.cc:4830] TS c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): tablet 2770cd77ebb642e3a52183a3871e33fa (table rebalancer_extra_table_14 [id=8827d1d86bff4223b2dc0d756546aa94]) successfully deleted
I20240426 09:50:36.002437 7274 consensus_queue.cc:237] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:36.004004 7144 raft_consensus.cc:1261] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:36.004060 9548 raft_consensus.cc:1261] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:36.004123 7208 consensus_peers.cc:485] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0a2aef96c3845b7bb6357b9914fe7f8. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:36.004357 29236 consensus_queue.cc:1035] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.004613 29040 consensus_queue.cc:1035] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.011523 29236 raft_consensus.cc:2932] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:36.013566 7144 raft_consensus.cc:2932] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:36.015092 9548 raft_consensus.cc:2932] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:36.016048 6735 catalog_manager.cc:5484] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:36.037398 9669 consensus_queue.cc:237] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:36.039017 9285 raft_consensus.cc:1261] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:36.039093 7013 raft_consensus.cc:1261] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:36.039181 9614 consensus_peers.cc:485] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab631877cfb646829d984aacd8a02795. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:36.039397 26842 consensus_queue.cc:1035] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.039610 29231 consensus_queue.cc:1035] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.041421 26842 raft_consensus.cc:2932] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:36.041810 7013 raft_consensus.cc:2932] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:36.044514 9285 raft_consensus.cc:2932] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:36.045696 6736 catalog_manager.cc:5484] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:36.072670 9415 consensus_queue.cc:237] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:36.074297 9285 raft_consensus.cc:1261] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:36.074411 7144 raft_consensus.cc:1261] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:36.074563 9352 consensus_peers.cc:485] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 957533c35bae4299b35b77ba9a465eb3. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:36.074827 29294 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:36.075217 29294 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:36.076952 28977 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:36.077436 7144 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:36.079907 9285 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:36.080677 6735 catalog_manager.cc:5484] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 3 to 4, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:36.103844 7274 consensus_queue.cc:237] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:36.105680 9669 raft_consensus.cc:1261] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:36.106103 9548 raft_consensus.cc:1261] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:36.106124 28823 consensus_queue.cc:1035] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240426 09:50:36.106319 7208 consensus_peers.cc:485] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3bd97a192183433aa48531ab7885e73e. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:36.106488 28823 consensus_queue.cc:1035] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.107993 28823 raft_consensus.cc:2932] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.108393 9549 raft_consensus.cc:2932] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.110036 9669 raft_consensus.cc:2932] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.111943 6736 catalog_manager.cc:5484] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.139238 9284 consensus_queue.cc:237] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:36.140995 7013 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:36.141086 9219 consensus_peers.cc:485] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: efa472b0759c4cbbb7db13dc3fbe0b50. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:36.141480 29403 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:36.142978 9549 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:36.143361 29403 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:36.145323 28406 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:36.145658 9549 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:36.147112 7013 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:36.149441 6736 catalog_manager.cc:5484] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:36.174593 9669 consensus_queue.cc:237] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:36.176231 9284 raft_consensus.cc:1261] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:36.176345 9614 consensus_peers.cc:485] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e494cf4a72945148ad4afa658ff1b03. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:36.176362 7274 raft_consensus.cc:1261] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:36.177008 28732 consensus_queue.cc:1035] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.179230 28732 consensus_queue.cc:1035] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.181468 28732 raft_consensus.cc:2932] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:36.183486 9284 raft_consensus.cc:2932] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:36.185302 6738 catalog_manager.cc:5484] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:36.187275 7275 raft_consensus.cc:2932] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:36.210938 9415 consensus_queue.cc:237] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:36.212584 9669 raft_consensus.cc:1261] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:36.212584 7144 raft_consensus.cc:1261] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:36.212709 9350 consensus_peers.cc:485] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 158666afa8f14791addcd6be64e6c562. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:36.212930 29294 consensus_queue.cc:1035] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.213255 28977 consensus_queue.cc:1035] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.215644 29294 raft_consensus.cc:2932] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.216012 7144 raft_consensus.cc:2932] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.218672 9669 raft_consensus.cc:2932] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.219552 6732 catalog_manager.cc:5484] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.243927 7013 consensus_queue.cc:237] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
W20240426 09:50:36.245536 6946 consensus_peers.cc:485] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 937837e55579473087c0d0fcd8bcb001. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:36.245518 7275 raft_consensus.cc:1261] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:36.245837 6882 raft_consensus.cc:1261] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:36.246133 29458 consensus_queue.cc:1035] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.246387 27516 consensus_queue.cc:1035] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:36.249732 29458 raft_consensus.cc:2932] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:36.250167 6882 raft_consensus.cc:2932] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:36.254891 7275 raft_consensus.cc:2932] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:36.255242 6738 catalog_manager.cc:5484] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:36.275413 9284 consensus_queue.cc:237] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:36.276945 6882 raft_consensus.cc:1261] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:36.276934 9415 raft_consensus.cc:1261] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:36.277338 29403 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:50:36.277519 9219 consensus_peers.cc:485] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab6450f48a36483883f8914075c224a8. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:36.277523 28406 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:36.279927 28406 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.280416 9415 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.282188 6882 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.285460 6738 catalog_manager.cc:5484] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 3 to 4, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:36.421844 29534 ts_tablet_manager.cc:902] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:50:36.422432 29534 tablet_copy_client.cc:308] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:50:36.422959 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:36.423406 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet c0a2aef96c3845b7bb6357b9914fe7f8 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:41659)
I20240426 09:50:36.423525 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet c0a2aef96c3845b7bb6357b9914fe7f8 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:41659: session id = 3b41d87e7bfb419f8aa7500100129b6c-c0a2aef96c3845b7bb6357b9914fe7f8
I20240426 09:50:36.424556 7295 tablet_copy_source_session.cc:205] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:36.428038 29534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0a2aef96c3845b7bb6357b9914fe7f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:36.430420 29534 tablet_copy_client.cc:783] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:36.430565 29534 tablet_copy_client.cc:647] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:36.432310 29534 tablet_copy_client.cc:523] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:36.434875 29534 tablet_bootstrap.cc:492] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:36.435761 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:36.437342 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:36.446295 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:36.446568 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:36.449139 29534 tablet_bootstrap.cc:492] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:36.449415 29534 tablet_bootstrap.cc:492] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:36.449627 29534 ts_tablet_manager.cc:1367] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20240426 09:50:36.450148 29534 raft_consensus.cc:348] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:36.450266 29534 raft_consensus.cc:729] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:36.450372 29534 consensus_queue.cc:260] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:36.451081 29534 ts_tablet_manager.cc:1398] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:36.452226 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:36.452347 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-c0a2aef96c3845b7bb6357b9914fe7f8 received from {username='slave'} at 127.0.99.134:41659
I20240426 09:50:36.452418 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-c0a2aef96c3845b7bb6357b9914fe7f8 on tablet c0a2aef96c3845b7bb6357b9914fe7f8 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:36.459160 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:36.469848 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:36.527597 29551 ts_tablet_manager.cc:902] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055)
I20240426 09:50:36.528205 29551 tablet_copy_client.cc:308] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.132:38055
I20240426 09:50:36.528663 7295 tablet_copy_service.cc:140] P e3394a4be9ea4def8aa39b333c82c5ff: Received BeginTabletCopySession request for tablet 3bd97a192183433aa48531ab7885e73e from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:36325)
I20240426 09:50:36.528785 7295 tablet_copy_service.cc:161] P e3394a4be9ea4def8aa39b333c82c5ff: Beginning new tablet copy session on tablet 3bd97a192183433aa48531ab7885e73e from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:36325: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-3bd97a192183433aa48531ab7885e73e
I20240426 09:50:36.529712 7295 tablet_copy_source_session.cc:205] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:36.530886 29551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd97a192183433aa48531ab7885e73e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:36.533177 29551 tablet_copy_client.cc:783] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:36.533326 29551 tablet_copy_client.cc:647] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:36.534740 29551 tablet_copy_client.cc:523] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:36.535925 29551 tablet_bootstrap.cc:492] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:36.540041 29551 tablet_bootstrap.cc:492] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:36.540292 29551 tablet_bootstrap.cc:492] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:36.540437 29551 ts_tablet_manager.cc:1367] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240426 09:50:36.540848 29551 raft_consensus.cc:348] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:36.540961 29551 raft_consensus.cc:729] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:36.541136 29551 consensus_queue.cc:260] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:36.542531 29551 ts_tablet_manager.cc:1398] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:50:36.543016 7295 tablet_copy_service.cc:342] P e3394a4be9ea4def8aa39b333c82c5ff: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-3bd97a192183433aa48531ab7885e73e received from {username='slave'} at 127.0.99.130:36325
I20240426 09:50:36.543107 7295 tablet_copy_service.cc:433] P e3394a4be9ea4def8aa39b333c82c5ff: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-3bd97a192183433aa48531ab7885e73e on tablet 3bd97a192183433aa48531ab7885e73e with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:36.583604 29588 ts_tablet_manager.cc:902] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:36.587317 29588 tablet_copy_client.cc:308] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:36.588253 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet ab631877cfb646829d984aacd8a02795 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:48359)
I20240426 09:50:36.588367 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet ab631877cfb646829d984aacd8a02795 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:48359: session id = 1aaf8befba984e199e96f53f46b50441-ab631877cfb646829d984aacd8a02795
I20240426 09:50:36.589277 9685 tablet_copy_source_session.cc:205] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:36.590098 29588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab631877cfb646829d984aacd8a02795. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:36.593801 29588 tablet_copy_client.cc:783] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:36.593971 29588 tablet_copy_client.cc:647] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:36.595369 29593 ts_tablet_manager.cc:902] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:36.596096 29593 tablet_copy_client.cc:308] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:36.596803 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet efa472b0759c4cbbb7db13dc3fbe0b50 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:36823)
I20240426 09:50:36.596908 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet efa472b0759c4cbbb7db13dc3fbe0b50 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:36823: session id = 1aaf8befba984e199e96f53f46b50441-efa472b0759c4cbbb7db13dc3fbe0b50
I20240426 09:50:36.597911 9306 tablet_copy_source_session.cc:205] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:36.599483 29588 tablet_copy_client.cc:523] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:36.601629 29588 tablet_bootstrap.cc:492] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:36.603078 29593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa472b0759c4cbbb7db13dc3fbe0b50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:36.605154 29593 tablet_copy_client.cc:783] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:36.614270 29588 tablet_bootstrap.cc:492] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:36.614434 29593 tablet_copy_client.cc:647] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:36.614694 29588 tablet_bootstrap.cc:492] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:36.614905 29588 ts_tablet_manager.cc:1367] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20240426 09:50:36.615346 29588 raft_consensus.cc:348] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:36.615478 29588 raft_consensus.cc:729] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:36.615629 29588 consensus_queue.cc:260] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:36.615998 29588 ts_tablet_manager.cc:1398] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:36.616789 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-ab631877cfb646829d984aacd8a02795 received from {username='slave'} at 127.0.99.131:48359
I20240426 09:50:36.616871 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-ab631877cfb646829d984aacd8a02795 on tablet ab631877cfb646829d984aacd8a02795 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:36.631659 29551 ts_tablet_manager.cc:902] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:36.632124 29551 tablet_copy_client.cc:308] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:36.633771 29534 ts_tablet_manager.cc:902] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:36.634194 29534 tablet_copy_client.cc:308] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.136:45059
I20240426 09:50:36.634581 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet 158666afa8f14791addcd6be64e6c562 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:58979)
I20240426 09:50:36.634677 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet 158666afa8f14791addcd6be64e6c562 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:58979: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-158666afa8f14791addcd6be64e6c562
I20240426 09:50:36.634869 9685 tablet_copy_service.cc:140] P 558c4ecd0ece460d9591aa7a3c799d19: Received BeginTabletCopySession request for tablet 7e494cf4a72945148ad4afa658ff1b03 from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:46185)
I20240426 09:50:36.634977 9685 tablet_copy_service.cc:161] P 558c4ecd0ece460d9591aa7a3c799d19: Beginning new tablet copy session on tablet 7e494cf4a72945148ad4afa658ff1b03 from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:46185: session id = 3b41d87e7bfb419f8aa7500100129b6c-7e494cf4a72945148ad4afa658ff1b03
I20240426 09:50:36.636013 9685 tablet_copy_source_session.cc:205] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:36.636858 29534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e494cf4a72945148ad4afa658ff1b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:36.637748 29593 tablet_copy_client.cc:523] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:36.639238 29593 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:36.639251 29534 tablet_copy_client.cc:783] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:36.639380 29534 tablet_copy_client.cc:647] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:36.639956 9437 tablet_copy_source_session.cc:205] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:36.641229 29551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 158666afa8f14791addcd6be64e6c562. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:36.643208 29551 tablet_copy_client.cc:783] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:36.643359 29551 tablet_copy_client.cc:647] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:36.644068 29593 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:36.644198 29608 ts_tablet_manager.cc:902] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203)
I20240426 09:50:36.644378 29593 tablet_bootstrap.cc:492] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:36.644532 29593 ts_tablet_manager.cc:1367] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20240426 09:50:36.644678 29608 tablet_copy_client.cc:308] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.134:37203
I20240426 09:50:36.646384 29593 raft_consensus.cc:348] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:36.646510 29593 raft_consensus.cc:729] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:36.646642 29593 consensus_queue.cc:260] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:36.646944 29593 ts_tablet_manager.cc:1398] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:50:36.647357 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-efa472b0759c4cbbb7db13dc3fbe0b50 received from {username='slave'} at 127.0.99.131:36823
I20240426 09:50:36.647439 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-efa472b0759c4cbbb7db13dc3fbe0b50 on tablet efa472b0759c4cbbb7db13dc3fbe0b50 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:36.654848 9437 tablet_copy_service.cc:140] P 3b41d87e7bfb419f8aa7500100129b6c: Received BeginTabletCopySession request for tablet 957533c35bae4299b35b77ba9a465eb3 from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:53025)
I20240426 09:50:36.654983 9437 tablet_copy_service.cc:161] P 3b41d87e7bfb419f8aa7500100129b6c: Beginning new tablet copy session on tablet 957533c35bae4299b35b77ba9a465eb3 from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:53025: session id = e3394a4be9ea4def8aa39b333c82c5ff-957533c35bae4299b35b77ba9a465eb3
I20240426 09:50:36.656090 9437 tablet_copy_source_session.cc:205] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:36.657353 29534 tablet_copy_client.cc:523] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:36.658905 29534 tablet_bootstrap.cc:492] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:36.659958 29551 tablet_copy_client.cc:523] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:36.661456 29551 tablet_bootstrap.cc:492] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:36.662508 29534 tablet_bootstrap.cc:492] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:36.662853 29534 tablet_bootstrap.cc:492] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:36.662917 29608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957533c35bae4299b35b77ba9a465eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:36.662997 29534 ts_tablet_manager.cc:1367] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20240426 09:50:36.663409 29534 raft_consensus.cc:348] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:36.663522 29534 raft_consensus.cc:729] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:36.663676 29534 consensus_queue.cc:260] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:36.663944 29534 ts_tablet_manager.cc:1398] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:36.666114 29608 tablet_copy_client.cc:783] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:36.666265 29608 tablet_copy_client.cc:647] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:36.667373 29551 tablet_bootstrap.cc:492] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:36.667635 29551 tablet_bootstrap.cc:492] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:36.668133 9685 tablet_copy_service.cc:342] P 558c4ecd0ece460d9591aa7a3c799d19: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-7e494cf4a72945148ad4afa658ff1b03 received from {username='slave'} at 127.0.99.134:46185
I20240426 09:50:36.668215 9685 tablet_copy_service.cc:433] P 558c4ecd0ece460d9591aa7a3c799d19: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-7e494cf4a72945148ad4afa658ff1b03 on tablet 7e494cf4a72945148ad4afa658ff1b03 with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:36.668542 29551 ts_tablet_manager.cc:1367] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240426 09:50:36.670034 29551 raft_consensus.cc:348] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:36.670441 29551 raft_consensus.cc:729] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:36.670915 29551 consensus_queue.cc:260] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:36.671232 29551 ts_tablet_manager.cc:1398] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:50:36.671751 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-158666afa8f14791addcd6be64e6c562 received from {username='slave'} at 127.0.99.130:58979
I20240426 09:50:36.671838 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-158666afa8f14791addcd6be64e6c562 on tablet 158666afa8f14791addcd6be64e6c562 with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:36.675146 29608 tablet_copy_client.cc:523] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:36.676578 29608 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:36.681196 29608 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:36.681463 29608 tablet_bootstrap.cc:492] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:36.681635 29608 ts_tablet_manager.cc:1367] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20240426 09:50:36.682232 29608 raft_consensus.cc:348] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:36.682361 29608 raft_consensus.cc:729] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:36.682523 29608 consensus_queue.cc:260] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:36.683286 29608 ts_tablet_manager.cc:1398] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20240426 09:50:36.683882 9437 tablet_copy_service.cc:342] P 3b41d87e7bfb419f8aa7500100129b6c: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-957533c35bae4299b35b77ba9a465eb3 received from {username='slave'} at 127.0.99.132:53025
I20240426 09:50:36.683971 9437 tablet_copy_service.cc:433] P 3b41d87e7bfb419f8aa7500100129b6c: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-957533c35bae4299b35b77ba9a465eb3 on tablet 957533c35bae4299b35b77ba9a465eb3 with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:36.718488 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:36.722074 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:36.723102 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:36.724108 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:36.726727 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:36.738010 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:36.747323 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:36.747984 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:36.764287 29627 ts_tablet_manager.cc:902] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:36.764811 29627 tablet_copy_client.cc:308] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:36.765355 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 937837e55579473087c0d0fcd8bcb001 from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:44679)
I20240426 09:50:36.765465 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 937837e55579473087c0d0fcd8bcb001 from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:44679: session id = 558c4ecd0ece460d9591aa7a3c799d19-937837e55579473087c0d0fcd8bcb001
I20240426 09:50:36.766470 7033 tablet_copy_source_session.cc:205] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:36.767446 29627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937837e55579473087c0d0fcd8bcb001. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:36.770550 29627 tablet_copy_client.cc:783] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:36.770699 29627 tablet_copy_client.cc:647] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:36.776225 29551 ts_tablet_manager.cc:902] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:36.776659 29551 tablet_copy_client.cc:308] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:36.777099 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet ab6450f48a36483883f8914075c224a8 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 ({username='slave'} at 127.0.99.130:39123)
I20240426 09:50:36.777223 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet ab6450f48a36483883f8914075c224a8 from peer 0a0bfe28bcc0435ebe9a54a9d6416899 at {username='slave'} at 127.0.99.130:39123: session id = 0a0bfe28bcc0435ebe9a54a9d6416899-ab6450f48a36483883f8914075c224a8
I20240426 09:50:36.778069 9306 tablet_copy_source_session.cc:205] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:36.778930 29551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6450f48a36483883f8914075c224a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:36.780987 29551 tablet_copy_client.cc:783] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:36.781134 29551 tablet_copy_client.cc:647] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:36.783954 29551 tablet_copy_client.cc:523] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:36.785221 29551 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap starting.
I20240426 09:50:36.807205 29627 tablet_copy_client.cc:523] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:36.808835 29627 tablet_bootstrap.cc:492] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:50:36.809407 29551 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:36.809684 29551 tablet_bootstrap.cc:492] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Bootstrap complete.
I20240426 09:50:36.809850 29551 ts_tablet_manager.cc:1367] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.000s
I20240426 09:50:36.810398 29551 raft_consensus.cc:348] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:36.810519 29551 raft_consensus.cc:729] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Initialized, Role: FOLLOWER
I20240426 09:50:36.810904 29551 consensus_queue.cc:260] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:36.812486 29551 ts_tablet_manager.cc:1398] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20240426 09:50:36.812747 29627 tablet_bootstrap.cc:492] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:36.813009 29627 tablet_bootstrap.cc:492] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:50:36.813172 29627 ts_tablet_manager.cc:1367] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.005s sys 0.000s
I20240426 09:50:36.813546 29627 raft_consensus.cc:348] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:36.813650 29627 raft_consensus.cc:729] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:50:36.813757 29627 consensus_queue.cc:260] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:36.814018 29627 ts_tablet_manager.cc:1398] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:36.815230 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-937837e55579473087c0d0fcd8bcb001 received from {username='slave'} at 127.0.99.136:44679
I20240426 09:50:36.815315 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-937837e55579473087c0d0fcd8bcb001 on tablet 937837e55579473087c0d0fcd8bcb001 with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:50:36.816993 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-ab6450f48a36483883f8914075c224a8 received from {username='slave'} at 127.0.99.130:39123
I20240426 09:50:36.817087 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session 0a0bfe28bcc0435ebe9a54a9d6416899-ab6450f48a36483883f8914075c224a8 on tablet ab6450f48a36483883f8914075c224a8 with peer 0a0bfe28bcc0435ebe9a54a9d6416899
I20240426 09:50:36.866564 9417 raft_consensus.cc:1203] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:36.949179 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:36.951896 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:36.961294 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:36.961737 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:36.982681 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:36.983189 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:36.983940 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:36.996688 7143 raft_consensus.cc:1203] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:37.000734 7144 raft_consensus.cc:1203] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:37.010118 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:37.055331 7011 raft_consensus.cc:1203] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:37.172319 7011 raft_consensus.cc:1203] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:37.182054 9417 raft_consensus.cc:1203] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:37.190382 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:37.194496 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:37.204970 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.205508 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.208171 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.232791 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:37.239902 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:37.245411 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:37.265859 7011 raft_consensus.cc:1203] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:37.271826 7273 raft_consensus.cc:1203] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:37.291759 9672 raft_consensus.cc:1203] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:37.424094 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:37.434115 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.438930 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.440737 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:37.440999 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.451020 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:37.453847 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:37.480165 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:37.528108 7273 consensus_queue.cc:237] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:37.529366 7011 raft_consensus.cc:1261] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:37.529438 9548 raft_consensus.cc:1261] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:37.529736 28823 consensus_queue.cc:1035] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:37.529896 29236 consensus_queue.cc:1035] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:37.531599 28823 raft_consensus.cc:2932] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:37.532274 9548 raft_consensus.cc:2932] T 3bd97a192183433aa48531ab7885e73e P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:37.536145 6738 catalog_manager.cc:5484] T 3bd97a192183433aa48531ab7885e73e P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:37.537906 7011 raft_consensus.cc:2932] T 3bd97a192183433aa48531ab7885e73e P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:37.541946 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd97a192183433aa48531ab7885e73e with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:37.543166 29819 tablet_replica.cc:330] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:37.543488 29819 raft_consensus.cc:2229] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:37.543749 29819 raft_consensus.cc:2258] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:37.545375 29819 ts_tablet_manager.cc:1875] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:37.547896 29819 ts_tablet_manager.cc:1888] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:37.548043 29819 log.cc:1196] T 3bd97a192183433aa48531ab7885e73e P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/3bd97a192183433aa48531ab7885e73e
I20240426 09:50:37.548802 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 3bd97a192183433aa48531ab7885e73e (table rebalancer_extra_table_70 [id=9ae9bbc423524c94a01adf462ebcd8bb]) successfully deleted
I20240426 09:50:37.686968 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:37.691308 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.701145 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.709224 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:37.711350 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.711964 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:37.712494 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:37.712687 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:37.797782 9415 consensus_queue.cc:237] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:37.799099 7144 raft_consensus.cc:1261] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:37.799167 7273 raft_consensus.cc:1261] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:37.799499 29294 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:37.799649 28726 consensus_queue.cc:1035] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:37.801630 28726 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:37.801898 7144 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:37.801898 7273 raft_consensus.cc:2932] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:37.806133 6734 catalog_manager.cc:5484] T 957533c35bae4299b35b77ba9a465eb3 P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 4 to 5, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:37.810377 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet 957533c35bae4299b35b77ba9a465eb3 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:37.818596 29898 tablet_replica.cc:330] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:37.818743 29898 raft_consensus.cc:2229] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:37.818931 29898 raft_consensus.cc:2258] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:37.819383 29898 ts_tablet_manager.cc:1875] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:37.821946 29898 ts_tablet_manager.cc:1888] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:37.822024 29898 log.cc:1196] T 957533c35bae4299b35b77ba9a465eb3 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/957533c35bae4299b35b77ba9a465eb3
I20240426 09:50:37.824739 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet 957533c35bae4299b35b77ba9a465eb3 (table rebalancer_extra_table_16 [id=7618178091bb40649d8fba1cc723cadc]) successfully deleted
I20240426 09:50:37.963852 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:37.966286 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.966660 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:37.980907 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:37.986883 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:37.992636 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:37.996104 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.012825 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:38.088238 9672 consensus_queue.cc:237] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:38.089381 7144 raft_consensus.cc:1261] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:38.089491 9284 raft_consensus.cc:1261] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:38.089795 29863 consensus_queue.cc:1035] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:38.089964 28732 consensus_queue.cc:1035] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:38.091630 29231 raft_consensus.cc:2932] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:38.091954 7144 raft_consensus.cc:2932] T ab631877cfb646829d984aacd8a02795 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:38.091954 9284 raft_consensus.cc:2932] T ab631877cfb646829d984aacd8a02795 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:38.096024 6736 catalog_manager.cc:5484] T ab631877cfb646829d984aacd8a02795 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:38.099256 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet ab631877cfb646829d984aacd8a02795 with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47654
I20240426 09:50:38.100855 29966 tablet_replica.cc:330] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:50:38.101006 29966 raft_consensus.cc:2229] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:38.101140 29966 raft_consensus.cc:2258] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:38.101533 29966 ts_tablet_manager.cc:1875] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:38.103915 29966 ts_tablet_manager.cc:1888] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:38.103991 29966 log.cc:1196] T ab631877cfb646829d984aacd8a02795 P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/ab631877cfb646829d984aacd8a02795
I20240426 09:50:38.104568 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet ab631877cfb646829d984aacd8a02795 (table rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]) successfully deleted
I20240426 09:50:38.263324 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:38.266322 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:38.267035 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:38.270742 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.275784 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.280814 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.281116 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:38.282943 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:38.361701 7273 consensus_queue.cc:237] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:38.363094 9415 raft_consensus.cc:1261] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:38.363092 7143 raft_consensus.cc:1261] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:38.363466 28823 consensus_queue.cc:1035] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:38.363821 29236 consensus_queue.cc:1035] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:38.366115 30039 raft_consensus.cc:2932] T c0a2aef96c3845b7bb6357b9914fe7f8 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:38.366452 9417 raft_consensus.cc:2932] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:38.366521 7143 raft_consensus.cc:2932] T c0a2aef96c3845b7bb6357b9914fe7f8 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:38.370008 6739 catalog_manager.cc:5484] T c0a2aef96c3845b7bb6357b9914fe7f8 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) evicted. New cstate: current_term: 1 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:38.373610 9527 tablet_service.cc:1514] Processing DeleteTablet for tablet c0a2aef96c3845b7bb6357b9914fe7f8 with delete_type TABLET_DATA_TOMBSTONED (TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47690
I20240426 09:50:38.374032 30040 tablet_replica.cc:330] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: stopping tablet replica
I20240426 09:50:38.374213 30040 raft_consensus.cc:2229] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:38.374365 30040 raft_consensus.cc:2258] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:38.375162 30040 ts_tablet_manager.cc:1875] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:38.377995 30040 ts_tablet_manager.cc:1888] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:38.378073 30040 log.cc:1196] T c0a2aef96c3845b7bb6357b9914fe7f8 P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/wals/c0a2aef96c3845b7bb6357b9914fe7f8
I20240426 09:50:38.378626 6724 catalog_manager.cc:4830] TS b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): tablet c0a2aef96c3845b7bb6357b9914fe7f8 (table rebalancer_extra_table_60 [id=0a40570d2a4c46d9a8f84468d97ae6ea]) successfully deleted
I20240426 09:50:38.534066 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:38.539250 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.545969 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.546280 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:38.548561 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.556180 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:38.556910 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:38.589670 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:38.656689 9284 consensus_queue.cc:237] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:38.657871 6882 raft_consensus.cc:1261] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:38.657874 7011 raft_consensus.cc:1261] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:38.658268 30106 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:38.658424 30104 consensus_queue.cc:1035] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:38.660187 30106 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:38.660511 6882 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:38.665127 6736 catalog_manager.cc:5484] T ab6450f48a36483883f8914075c224a8 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:38.665360 7011 raft_consensus.cc:2932] T ab6450f48a36483883f8914075c224a8 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:38.668738 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet ab6450f48a36483883f8914075c224a8 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:38.669724 30130 tablet_replica.cc:330] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:38.669864 30130 raft_consensus.cc:2229] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:38.670010 30130 raft_consensus.cc:2258] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:38.670455 30130 ts_tablet_manager.cc:1875] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:38.672794 30130 ts_tablet_manager.cc:1888] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:38.672875 30130 log.cc:1196] T ab6450f48a36483883f8914075c224a8 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/ab6450f48a36483883f8914075c224a8
I20240426 09:50:38.673584 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet ab6450f48a36483883f8914075c224a8 (table rebalancer_extra_table_22 [id=ad1adf5b84bb493e971dc811450c9be1]) successfully deleted
I20240426 09:50:38.806524 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:38.806962 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.813997 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:38.828119 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:38.831224 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:38.836999 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.862944 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:38.864388 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:38.927970 9417 consensus_queue.cc:237] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } }
I20240426 09:50:38.929309 7143 raft_consensus.cc:1261] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:38.929319 7011 raft_consensus.cc:1261] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:38.929812 29893 consensus_queue.cc:1035] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:38.930003 29294 consensus_queue.cc:1035] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:38.931885 30196 raft_consensus.cc:2932] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:38.932161 7011 raft_consensus.cc:2932] T 158666afa8f14791addcd6be64e6c562 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:38.933635 7143 raft_consensus.cc:2932] T 158666afa8f14791addcd6be64e6c562 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:38.937680 6732 catalog_manager.cc:5484] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } }
I20240426 09:50:38.942885 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 158666afa8f14791addcd6be64e6c562 with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:38.943199 30202 tablet_replica.cc:330] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:38.943352 30202 raft_consensus.cc:2229] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:38.943529 30202 raft_consensus.cc:2258] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:38.944010 30202 ts_tablet_manager.cc:1875] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:38.946534 30202 ts_tablet_manager.cc:1888] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:38.946609 30202 log.cc:1196] T 158666afa8f14791addcd6be64e6c562 P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/158666afa8f14791addcd6be64e6c562
I20240426 09:50:38.947276 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 158666afa8f14791addcd6be64e6c562 (table rebalancer_extra_table_64 [id=d219866265244cc2bf7a2b9a6370c518]) successfully deleted
I20240426 09:50:39.093681 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:39.095949 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:39.098722 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.102970 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:39.105984 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.120990 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:39.132717 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.139626 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:39.208516 9673 consensus_queue.cc:237] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:39.209830 7011 raft_consensus.cc:1261] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:50:39.209966 7273 raft_consensus.cc:1261] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240426 09:50:39.210402 29231 consensus_queue.cc:1035] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:39.210548 28732 consensus_queue.cc:1035] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:39.212293 29231 raft_consensus.cc:2932] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.212792 7011 raft_consensus.cc:2932] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.213021 7274 raft_consensus.cc:2932] T 5b66a5a6cf3348ffbaa56679985e9aa2 P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.216002 6738 catalog_manager.cc:5484] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 2 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 2 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.218732 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b66a5a6cf3348ffbaa56679985e9aa2 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:39.219369 30270 tablet_replica.cc:330] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:39.219574 30270 raft_consensus.cc:2229] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:39.219748 30270 raft_consensus.cc:2258] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:39.220369 30270 ts_tablet_manager.cc:1875] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:39.223937 30270 ts_tablet_manager.cc:1888] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240426 09:50:39.224035 30270 log.cc:1196] T 5b66a5a6cf3348ffbaa56679985e9aa2 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/5b66a5a6cf3348ffbaa56679985e9aa2
I20240426 09:50:39.224792 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 5b66a5a6cf3348ffbaa56679985e9aa2 (table rebalancer_extra_table_72 [id=088796f182714fd9bc46e4cafea5fd56]) successfully deleted
I20240426 09:50:39.251102 9284 consensus_queue.cc:237] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:39.252723 7274 raft_consensus.cc:1261] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:39.252708 7143 raft_consensus.cc:1261] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:39.253123 30298 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240426 09:50:39.253331 9219 consensus_peers.cc:485] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 279519370ee640fd9bbd3028008f1d5d. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:39.253331 30106 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:39.255103 30298 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.255524 7143 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.257231 7274 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.260260 6731 catalog_manager.cc:5484] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 reported cstate change: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.397344 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.398186 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.402982 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:39.405846 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:39.411489 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:39.412585 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.426924 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:39.444231 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:39.506089 7011 consensus_queue.cc:237] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:39.507367 6880 raft_consensus.cc:1261] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:39.507405 9673 raft_consensus.cc:1261] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:39.507746 29727 consensus_queue.cc:1035] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:39.507926 30105 consensus_queue.cc:1035] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:39.509774 30105 raft_consensus.cc:2932] T 937837e55579473087c0d0fcd8bcb001 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.510212 9673 raft_consensus.cc:2932] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.510321 6880 raft_consensus.cc:2932] T 937837e55579473087c0d0fcd8bcb001 P c244e02f3cf847c0a5b69e1059f1071c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.513522 6739 catalog_manager.cc:5484] T 937837e55579473087c0d0fcd8bcb001 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:39.516906 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 937837e55579473087c0d0fcd8bcb001 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:39.517510 30376 tablet_replica.cc:330] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:39.517669 30376 raft_consensus.cc:2229] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:39.517798 30376 raft_consensus.cc:2258] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:39.518285 30376 ts_tablet_manager.cc:1875] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:39.520730 30376 ts_tablet_manager.cc:1888] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:39.520836 30376 log.cc:1196] T 937837e55579473087c0d0fcd8bcb001 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/937837e55579473087c0d0fcd8bcb001
I20240426 09:50:39.521711 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 937837e55579473087c0d0fcd8bcb001 (table rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]) successfully deleted
I20240426 09:50:39.690341 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.693711 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:39.694372 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:39.703846 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:39.706930 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:39.708004 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.714659 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.732403 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:39.755702 30427 ts_tablet_manager.cc:902] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545)
I20240426 09:50:39.759168 30427 tablet_copy_client.cc:308] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.133:40545
I20240426 09:50:39.759744 9306 tablet_copy_service.cc:140] P a692602e1a2e4794877e6fb92a7b9962: Received BeginTabletCopySession request for tablet 279519370ee640fd9bbd3028008f1d5d from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:50877)
I20240426 09:50:39.759850 9306 tablet_copy_service.cc:161] P a692602e1a2e4794877e6fb92a7b9962: Beginning new tablet copy session on tablet 279519370ee640fd9bbd3028008f1d5d from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:50877: session id = 558c4ecd0ece460d9591aa7a3c799d19-279519370ee640fd9bbd3028008f1d5d
I20240426 09:50:39.760731 9306 tablet_copy_source_session.cc:205] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:39.761912 30427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279519370ee640fd9bbd3028008f1d5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:39.764098 30427 tablet_copy_client.cc:783] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:39.764259 30427 tablet_copy_client.cc:647] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:39.765350 30427 tablet_copy_client.cc:523] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:39.766920 30427 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:50:39.771435 30427 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:39.773401 30427 tablet_bootstrap.cc:492] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:50:39.773594 30427 ts_tablet_manager.cc:1367] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240426 09:50:39.774081 30427 raft_consensus.cc:348] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:39.774209 30427 raft_consensus.cc:729] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:50:39.774323 30427 consensus_queue.cc:260] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:39.774641 30427 ts_tablet_manager.cc:1398] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:39.776074 9306 tablet_copy_service.cc:342] P a692602e1a2e4794877e6fb92a7b9962: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-279519370ee640fd9bbd3028008f1d5d received from {username='slave'} at 127.0.99.136:50877
I20240426 09:50:39.776161 9306 tablet_copy_service.cc:433] P a692602e1a2e4794877e6fb92a7b9962: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-279519370ee640fd9bbd3028008f1d5d on tablet 279519370ee640fd9bbd3028008f1d5d with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:50:39.793371 9672 consensus_queue.cc:237] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:39.794668 9417 raft_consensus.cc:1261] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:39.794673 9285 raft_consensus.cc:1261] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer 558c4ecd0ece460d9591aa7a3c799d19: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:39.795156 30269 consensus_queue.cc:1035] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:39.795401 30268 consensus_queue.cc:1035] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:39.797183 30268 raft_consensus.cc:2932] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:39.798362 9417 raft_consensus.cc:2932] T 7e494cf4a72945148ad4afa658ff1b03 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:39.801088 9285 raft_consensus.cc:2932] T 7e494cf4a72945148ad4afa658ff1b03 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:39.801198 6733 catalog_manager.cc:5484] T 7e494cf4a72945148ad4afa658ff1b03 P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "558c4ecd0ece460d9591aa7a3c799d19" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:39.809129 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e494cf4a72945148ad4afa658ff1b03 with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:39.809489 30376 tablet_replica.cc:330] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:39.809631 30376 raft_consensus.cc:2229] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:39.809751 30376 raft_consensus.cc:2258] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:39.810209 30376 ts_tablet_manager.cc:1875] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:39.812378 30376 ts_tablet_manager.cc:1888] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:39.812453 30376 log.cc:1196] T 7e494cf4a72945148ad4afa658ff1b03 P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/7e494cf4a72945148ad4afa658ff1b03
I20240426 09:50:39.812929 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 7e494cf4a72945148ad4afa658ff1b03 (table rebalancer_extra_table_66 [id=3d5b24d1e0cd45ed91b90e25d7504c1d]) successfully deleted
I20240426 09:50:39.950675 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:39.952010 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:39.955549 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:39.958338 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.960131 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.982931 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:39.993779 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:40.000655 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:40.051338 9284 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "efa472b0759c4cbbb7db13dc3fbe0b50"
dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:56426
I20240426 09:50:40.051520 9284 raft_consensus.cc:595] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:40.179585 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:40.182027 7123 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:40.183677 9264 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:40.207031 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:40.213929 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:40.217463 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:40.231812 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:40.236346 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:40.267216 30298 raft_consensus.cc:979] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: : Instructing follower 1aaf8befba984e199e96f53f46b50441 to start an election
I20240426 09:50:40.267329 30298 raft_consensus.cc:1067] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Signalling peer 1aaf8befba984e199e96f53f46b50441 to start an election
I20240426 09:50:40.267887 7144 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "efa472b0759c4cbbb7db13dc3fbe0b50"
dest_uuid: "1aaf8befba984e199e96f53f46b50441"
from {username='slave'} at 127.0.99.133:54629
I20240426 09:50:40.268057 7144 raft_consensus.cc:482] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:40.268116 7144 raft_consensus.cc:3037] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:40.269295 7144 raft_consensus.cc:504] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:40.269845 7144 leader_election.cc:288] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 2 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:40.270416 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa472b0759c4cbbb7db13dc3fbe0b50" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:50:40.270438 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa472b0759c4cbbb7db13dc3fbe0b50" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:50:40.270535 7013 raft_consensus.cc:3037] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:40.270565 9549 raft_consensus.cc:3037] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:40.271591 7013 raft_consensus.cc:2454] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 2.
I20240426 09:50:40.271592 9549 raft_consensus.cc:2454] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 2.
I20240426 09:50:40.272537 7077 leader_election.cc:302] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 1aaf8befba984e199e96f53f46b50441, b8e863bc0edf4940bbf3e9f9b2bbb3e5; no voters:
I20240426 09:50:40.272697 29728 raft_consensus.cc:2781] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:40.272758 29728 raft_consensus.cc:686] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [term 2 LEADER]: Becoming Leader. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Running, Role: LEADER
I20240426 09:50:40.272876 29728 consensus_queue.cc:237] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:40.274102 9284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa472b0759c4cbbb7db13dc3fbe0b50" candidate_uuid: "1aaf8befba984e199e96f53f46b50441" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a692602e1a2e4794877e6fb92a7b9962"
I20240426 09:50:40.274221 9284 raft_consensus.cc:3032] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:40.274257 9284 raft_consensus.cc:729] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a692602e1a2e4794877e6fb92a7b9962, State: Running, Role: LEADER
I20240426 09:50:40.274401 9284 consensus_queue.cc:260] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:40.274595 9284 raft_consensus.cc:3037] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:40.275993 9284 raft_consensus.cc:2454] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aaf8befba984e199e96f53f46b50441 in term 2.
I20240426 09:50:40.276947 6736 catalog_manager.cc:5484] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 reported cstate change: term changed from 1 to 2, leader changed from a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) to 1aaf8befba984e199e96f53f46b50441 (127.0.99.131). New cstate: current_term: 2 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:40.287132 9672 raft_consensus.cc:1203] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:40.456518 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:40.458189 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:40.465328 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:40.466001 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:40.467887 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:40.468904 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:40.474933 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:40.481415 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:40.555920 9284 consensus_queue.cc:237] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:40.557294 9672 raft_consensus.cc:1261] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:40.557682 7144 raft_consensus.cc:1261] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer a692602e1a2e4794877e6fb92a7b9962: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:40.557718 30297 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:40.558307 30297 consensus_queue.cc:1035] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:40.559373 30649 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P a692602e1a2e4794877e6fb92a7b9962 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:40.559763 9672 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:40.562660 7144 raft_consensus.cc:2932] T 279519370ee640fd9bbd3028008f1d5d P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:40.563203 6738 catalog_manager.cc:5484] T 279519370ee640fd9bbd3028008f1d5d P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 4 to 5, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) evicted. New cstate: current_term: 1 leader_uuid: "a692602e1a2e4794877e6fb92a7b9962" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:40.569905 7254 tablet_service.cc:1514] Processing DeleteTablet for tablet 279519370ee640fd9bbd3028008f1d5d with delete_type TABLET_DATA_TOMBSTONED (TS e3394a4be9ea4def8aa39b333c82c5ff not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44728
I20240426 09:50:40.574951 30657 tablet_replica.cc:330] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff: stopping tablet replica
I20240426 09:50:40.575114 30657 raft_consensus.cc:2229] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:40.575277 30657 raft_consensus.cc:2258] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:40.575810 30657 ts_tablet_manager.cc:1875] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:40.578630 30657 ts_tablet_manager.cc:1888] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:40.578716 30657 log.cc:1196] T 279519370ee640fd9bbd3028008f1d5d P e3394a4be9ea4def8aa39b333c82c5ff: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/wals/279519370ee640fd9bbd3028008f1d5d
I20240426 09:50:40.579368 6726 catalog_manager.cc:4830] TS e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): tablet 279519370ee640fd9bbd3028008f1d5d (table rebalancer_extra_table_12 [id=614585ed1c454f9a8be0072b429b3546]) successfully deleted
I20240426 09:50:40.618261 7144 consensus_queue.cc:237] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:40.619853 7013 raft_consensus.cc:1261] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:40.619895 9549 raft_consensus.cc:1261] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240426 09:50:40.619968 7077 consensus_peers.cc:485] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecd7063af4854e17bb2211e31033a89b. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:40.620195 30689 consensus_queue.cc:1035] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:40.621438 29728 consensus_queue.cc:1035] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:40.623374 30689 raft_consensus.cc:2932] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:40.623806 7013 raft_consensus.cc:2932] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:40.626981 9549 raft_consensus.cc:2932] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:40.627679 6738 catalog_manager.cc:5484] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index -1 to 2, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:40.650930 7013 consensus_queue.cc:237] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
W20240426 09:50:40.652642 6948 consensus_peers.cc:485] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14752ab4ee9c4389b9d7afbfe1f5534c. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:40.652895 9672 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:40.653244 30708 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:40.654013 9417 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:40.654623 29727 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240426 09:50:40.656332 30105 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:40.656951 9417 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:40.658792 9672 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:40.660403 6737 catalog_manager.cc:5484] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: config changed from index -1 to 2, VOTER e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:40.759848 7013 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:40.760370 28966 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:40.790480 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:40.791759 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:40.812500 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:40.821033 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:40.823238 9285 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:40.824123 28966 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:40.829366 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:40.832204 9549 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:40.832600 30689 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:40.844949 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:40.851075 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:40.875190 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:41.042596 30812 ts_tablet_manager.cc:902] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021)
I20240426 09:50:41.043200 30812 tablet_copy_client.cc:308] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.131:39021
I20240426 09:50:41.043756 7164 tablet_copy_service.cc:140] P 1aaf8befba984e199e96f53f46b50441: Received BeginTabletCopySession request for tablet ecd7063af4854e17bb2211e31033a89b from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:32985)
I20240426 09:50:41.043902 7164 tablet_copy_service.cc:161] P 1aaf8befba984e199e96f53f46b50441: Beginning new tablet copy session on tablet ecd7063af4854e17bb2211e31033a89b from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:32985: session id = 558c4ecd0ece460d9591aa7a3c799d19-ecd7063af4854e17bb2211e31033a89b
I20240426 09:50:41.044953 7164 tablet_copy_source_session.cc:205] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:41.046116 30812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecd7063af4854e17bb2211e31033a89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:41.053303 30812 tablet_copy_client.cc:783] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:41.053511 30812 tablet_copy_client.cc:647] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:41.060009 30812 tablet_copy_client.cc:523] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:41.063650 30812 tablet_bootstrap.cc:492] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:50:41.064496 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:41.067621 30812 tablet_bootstrap.cc:492] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:41.067895 30812 tablet_bootstrap.cc:492] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:50:41.068049 30812 ts_tablet_manager.cc:1367] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20240426 09:50:41.072580 30812 raft_consensus.cc:348] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:41.072717 30812 raft_consensus.cc:729] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:50:41.073663 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:41.074863 30812 consensus_queue.cc:260] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:41.075227 30812 ts_tablet_manager.cc:1398] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:50:41.075773 7164 tablet_copy_service.cc:342] P 1aaf8befba984e199e96f53f46b50441: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-ecd7063af4854e17bb2211e31033a89b received from {username='slave'} at 127.0.99.136:32985
I20240426 09:50:41.075870 7164 tablet_copy_service.cc:433] P 1aaf8befba984e199e96f53f46b50441: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-ecd7063af4854e17bb2211e31033a89b on tablet ecd7063af4854e17bb2211e31033a89b with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:50:41.080791 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:41.083119 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:41.083673 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:41.083774 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:41.084161 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:41.101318 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:41.150400 30830 ts_tablet_manager.cc:902] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:41.150947 30830 tablet_copy_client.cc:308] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:41.151484 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 14752ab4ee9c4389b9d7afbfe1f5534c from peer e3394a4be9ea4def8aa39b333c82c5ff ({username='slave'} at 127.0.99.132:32839)
I20240426 09:50:41.151599 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 14752ab4ee9c4389b9d7afbfe1f5534c from peer e3394a4be9ea4def8aa39b333c82c5ff at {username='slave'} at 127.0.99.132:32839: session id = e3394a4be9ea4def8aa39b333c82c5ff-14752ab4ee9c4389b9d7afbfe1f5534c
I20240426 09:50:41.152642 7033 tablet_copy_source_session.cc:205] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:41.154680 30830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14752ab4ee9c4389b9d7afbfe1f5534c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:41.157070 30830 tablet_copy_client.cc:783] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:41.157224 30830 tablet_copy_client.cc:647] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:41.158823 30830 tablet_copy_client.cc:523] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:41.160513 30830 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap starting.
I20240426 09:50:41.165073 30830 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:41.165355 30830 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: Bootstrap complete.
I20240426 09:50:41.165514 30830 ts_tablet_manager.cc:1367] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:50:41.166005 30830 raft_consensus.cc:348] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:41.166112 30830 raft_consensus.cc:729] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Initialized, Role: FOLLOWER
I20240426 09:50:41.166237 30830 consensus_queue.cc:260] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:41.166491 30830 ts_tablet_manager.cc:1398] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:41.166931 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-14752ab4ee9c4389b9d7afbfe1f5534c received from {username='slave'} at 127.0.99.132:32839
I20240426 09:50:41.167007 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session e3394a4be9ea4def8aa39b333c82c5ff-14752ab4ee9c4389b9d7afbfe1f5534c on tablet 14752ab4ee9c4389b9d7afbfe1f5534c with peer e3394a4be9ea4def8aa39b333c82c5ff
I20240426 09:50:41.325976 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:41.330154 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:41.331072 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:41.332073 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:41.333119 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:41.335731 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:41.338925 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:41.347954 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:41.429275 7144 consensus_queue.cc:237] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:41.430275 7013 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:41.430627 9549 raft_consensus.cc:1261] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:41.430936 29728 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:41.431074 30689 consensus_queue.cc:1035] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:41.432581 30915 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P 1aaf8befba984e199e96f53f46b50441 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:41.432880 9549 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:41.432888 7013 raft_consensus.cc:2932] T efa472b0759c4cbbb7db13dc3fbe0b50 P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:41.436556 6739 catalog_manager.cc:5484] T efa472b0759c4cbbb7db13dc3fbe0b50 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 reported cstate change: config changed from index 4 to 6, VOTER a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133) evicted. New cstate: current_term: 2 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:41.441113 9264 tablet_service.cc:1514] Processing DeleteTablet for tablet efa472b0759c4cbbb7db13dc3fbe0b50 with delete_type TABLET_DATA_TOMBSTONED (TS a692602e1a2e4794877e6fb92a7b9962 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:32806
I20240426 09:50:41.441500 30924 tablet_replica.cc:330] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: stopping tablet replica
I20240426 09:50:41.441679 30924 raft_consensus.cc:2229] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:41.441854 30924 raft_consensus.cc:2258] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:41.442893 30924 ts_tablet_manager.cc:1875] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:41.445470 30924 ts_tablet_manager.cc:1888] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:50:41.445685 30924 log.cc:1196] T efa472b0759c4cbbb7db13dc3fbe0b50 P a692602e1a2e4794877e6fb92a7b9962: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/wals/efa472b0759c4cbbb7db13dc3fbe0b50
I20240426 09:50:41.446731 6725 catalog_manager.cc:4830] TS a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): tablet efa472b0759c4cbbb7db13dc3fbe0b50 (table rebalancer_extra_table_52 [id=7142db15f15f4842baf6223c339353a3]) successfully deleted
I20240426 09:50:41.549607 7274 raft_consensus.cc:1203] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:41.574237 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:41.578613 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:41.585578 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:41.585768 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:41.589331 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:41.589509 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:41.605036 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:41.613839 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:41.640914 9673 raft_consensus.cc:1203] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240426 09:50:41.688995 7013 consensus_queue.cc:237] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } }
I20240426 09:50:41.690215 7274 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:41.690372 9416 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:41.690537 29727 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:41.690791 30711 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:41.692497 30995 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:41.692858 9416 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:41.692869 7274 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:41.696924 6732 catalog_manager.cc:5484] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 2 to 3, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) evicted. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } }
I20240426 09:50:41.701033 9660 tablet_service.cc:1514] Processing DeleteTablet for tablet 14752ab4ee9c4389b9d7afbfe1f5534c with delete_type TABLET_DATA_TOMBSTONED (TS 558c4ecd0ece460d9591aa7a3c799d19 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:33488
I20240426 09:50:41.701695 31001 tablet_replica.cc:330] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19: stopping tablet replica
I20240426 09:50:41.701902 31001 raft_consensus.cc:2229] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:41.702083 31001 raft_consensus.cc:2258] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:41.702570 31001 ts_tablet_manager.cc:1875] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:41.705035 31001 ts_tablet_manager.cc:1888] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:41.705102 31001 log.cc:1196] T 14752ab4ee9c4389b9d7afbfe1f5534c P 558c4ecd0ece460d9591aa7a3c799d19: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/wals/14752ab4ee9c4389b9d7afbfe1f5534c
I20240426 09:50:41.705765 6724 catalog_manager.cc:4830] TS 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): tablet 14752ab4ee9c4389b9d7afbfe1f5534c (table rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]) successfully deleted
I20240426 09:50:41.833950 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:41.845427 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:41.850934 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:41.853055 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:41.862211 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:41.864527 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:41.865195 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:41.894838 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:41.966178 7144 consensus_queue.cc:237] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:41.967564 9673 raft_consensus.cc:1261] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:41.967645 9549 raft_consensus.cc:1261] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Refusing update from remote peer 1aaf8befba984e199e96f53f46b50441: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:41.967909 30689 consensus_queue.cc:1035] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:41.968108 29728 consensus_queue.cc:1035] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240426 09:50:41.969715 29728 raft_consensus.cc:2932] T ecd7063af4854e17bb2211e31033a89b P 1aaf8befba984e199e96f53f46b50441 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:41.970031 9673 raft_consensus.cc:2932] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:41.971881 9549 raft_consensus.cc:2932] T ecd7063af4854e17bb2211e31033a89b P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:41.974923 6735 catalog_manager.cc:5484] T ecd7063af4854e17bb2211e31033a89b P 558c4ecd0ece460d9591aa7a3c799d19 reported cstate change: config changed from index 2 to 3, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 1 leader_uuid: "1aaf8befba984e199e96f53f46b50441" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:41.977741 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet ecd7063af4854e17bb2211e31033a89b with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:47654
I20240426 09:50:41.978572 31072 tablet_replica.cc:330] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:50:41.978792 31072 raft_consensus.cc:2229] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:41.979038 31072 raft_consensus.cc:2258] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:41.979579 31072 ts_tablet_manager.cc:1875] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:41.982215 31072 ts_tablet_manager.cc:1888] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240426 09:50:41.982283 31072 log.cc:1196] T ecd7063af4854e17bb2211e31033a89b P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/ecd7063af4854e17bb2211e31033a89b
I20240426 09:50:41.983233 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet ecd7063af4854e17bb2211e31033a89b (table rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]) successfully deleted
I20240426 09:50:42.181300 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:42.183347 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:42.202155 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:42.202188 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:42.202355 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:42.203015 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:42.204768 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:42.205150 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:42.315222 7013 consensus_queue.cc:237] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:42.316673 7275 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:42.316826 9416 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:42.316957 6946 consensus_peers.cc:485] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979): Couldn't send request to peer b8e863bc0edf4940bbf3e9f9b2bbb3e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14752ab4ee9c4389b9d7afbfe1f5534c. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:42.317085 30995 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:42.317283 30711 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:42.319067 30711 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:42.321594 9416 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:42.323115 7275 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:42.324434 6739 catalog_manager.cc:5484] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 3 to 4, VOTER b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:42.346426 7013 consensus_queue.cc:237] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:42.348067 7275 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:42.348078 7143 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer 0a0bfe28bcc0435ebe9a54a9d6416899: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:42.348124 6946 consensus_peers.cc:485] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 -> Peer 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059): Couldn't send request to peer 558c4ecd0ece460d9591aa7a3c799d19. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7799bbe0befb41d0a155ff51f41c23ad. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:42.348477 30995 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:42.348695 29727 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:42.351943 29727 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:42.353782 7275 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:42.356395 6739 catalog_manager.cc:5484] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 reported cstate change: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New cstate: current_term: 1 leader_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:42.357221 7143 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:42.378024 6882 consensus_queue.cc:237] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:42.379487 9284 raft_consensus.cc:1261] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:42.379837 6815 consensus_peers.cc:485] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e78db4ae5cd4c40a3ed0478f8e9dee6. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:42.380494 30899 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:42.383067 9416 raft_consensus.cc:1261] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:42.383459 31182 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:42.385525 30092 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:42.386181 9416 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:42.389166 9284 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:42.390093 6736 catalog_manager.cc:5484] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 3 to 4, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:42.416718 6882 consensus_queue.cc:237] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:42.418226 7143 raft_consensus.cc:1261] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240426 09:50:42.418212 9284 raft_consensus.cc:1261] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240426 09:50:42.418337 6815 consensus_peers.cc:485] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c -> Peer 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): Couldn't send request to peer 3b41d87e7bfb419f8aa7500100129b6c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1cec7dfa2ba1466d9d5fc6e55baf280b. This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:42.418669 31185 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:42.418869 30899 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:50:42.421294 31185 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:42.421708 9284 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:42.425384 6739 catalog_manager.cc:5484] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 reported cstate change: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:42.427359 7143 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:42.556362 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:42.561734 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:42.565456 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:42.586855 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:42.588464 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:42.591213 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:42.592319 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:42.617614 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:42.711462 31297 ts_tablet_manager.cc:902] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:42.712090 31297 tablet_copy_client.cc:308] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:42.712690 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 14752ab4ee9c4389b9d7afbfe1f5534c from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 ({username='slave'} at 127.0.99.135:42099)
I20240426 09:50:42.712805 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 14752ab4ee9c4389b9d7afbfe1f5534c from peer b8e863bc0edf4940bbf3e9f9b2bbb3e5 at {username='slave'} at 127.0.99.135:42099: session id = b8e863bc0edf4940bbf3e9f9b2bbb3e5-14752ab4ee9c4389b9d7afbfe1f5534c
I20240426 09:50:42.713860 7033 tablet_copy_source_session.cc:205] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:42.715973 31297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14752ab4ee9c4389b9d7afbfe1f5534c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:42.719314 31297 tablet_copy_client.cc:783] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:42.719486 31297 tablet_copy_client.cc:647] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:42.720744 31297 tablet_copy_client.cc:523] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:42.722224 31297 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap starting.
I20240426 09:50:42.726617 31297 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:42.726874 31297 tablet_bootstrap.cc:492] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Bootstrap complete.
I20240426 09:50:42.727030 31297 ts_tablet_manager.cc:1367] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.002s
I20240426 09:50:42.727541 31297 raft_consensus.cc:348] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:42.727666 31297 raft_consensus.cc:729] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e863bc0edf4940bbf3e9f9b2bbb3e5, State: Initialized, Role: FOLLOWER
I20240426 09:50:42.727800 31297 consensus_queue.cc:260] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:42.729854 31297 ts_tablet_manager.cc:1398] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:50:42.731056 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-14752ab4ee9c4389b9d7afbfe1f5534c received from {username='slave'} at 127.0.99.135:42099
I20240426 09:50:42.731138 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session b8e863bc0edf4940bbf3e9f9b2bbb3e5-14752ab4ee9c4389b9d7afbfe1f5534c on tablet 14752ab4ee9c4389b9d7afbfe1f5534c with peer b8e863bc0edf4940bbf3e9f9b2bbb3e5
I20240426 09:50:42.793435 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:42.799383 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:42.799775 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:42.800871 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:42.804313 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:42.805394 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:42.809645 31316 ts_tablet_manager.cc:902] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:42.810322 31316 tablet_copy_client.cc:308] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:42.814864 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 from peer 1aaf8befba984e199e96f53f46b50441 ({username='slave'} at 127.0.99.131:53629)
I20240426 09:50:42.814986 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 from peer 1aaf8befba984e199e96f53f46b50441 at {username='slave'} at 127.0.99.131:53629: session id = 1aaf8befba984e199e96f53f46b50441-2e78db4ae5cd4c40a3ed0478f8e9dee6
I20240426 09:50:42.815976 6902 tablet_copy_source_session.cc:205] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:42.819119 31316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:42.821311 31316 tablet_copy_client.cc:783] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:42.821466 31316 tablet_copy_client.cc:647] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:42.842840 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:42.850456 31316 tablet_copy_client.cc:523] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:42.852144 31316 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: Bootstrap starting.
I20240426 09:50:42.861246 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:42.868589 31316 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:42.868896 31316 tablet_bootstrap.cc:492] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: Bootstrap complete.
I20240426 09:50:42.869060 31316 ts_tablet_manager.cc:1367] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20240426 09:50:42.869925 31316 raft_consensus.cc:348] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:42.870064 31316 raft_consensus.cc:729] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aaf8befba984e199e96f53f46b50441, State: Initialized, Role: FOLLOWER
I20240426 09:50:42.870231 31316 consensus_queue.cc:260] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:42.870550 31316 ts_tablet_manager.cc:1398] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:42.872049 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 1aaf8befba984e199e96f53f46b50441-2e78db4ae5cd4c40a3ed0478f8e9dee6 received from {username='slave'} at 127.0.99.131:53629
I20240426 09:50:42.872126 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 1aaf8befba984e199e96f53f46b50441-2e78db4ae5cd4c40a3ed0478f8e9dee6 on tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 with peer 1aaf8befba984e199e96f53f46b50441
I20240426 09:50:42.913733 31333 ts_tablet_manager.cc:902] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: Initiating tablet copy from peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571)
I20240426 09:50:42.914243 31333 tablet_copy_client.cc:308] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.129:41571
I20240426 09:50:42.914710 6902 tablet_copy_service.cc:140] P c244e02f3cf847c0a5b69e1059f1071c: Received BeginTabletCopySession request for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b from peer 3b41d87e7bfb419f8aa7500100129b6c ({username='slave'} at 127.0.99.134:40097)
I20240426 09:50:42.914877 6902 tablet_copy_service.cc:161] P c244e02f3cf847c0a5b69e1059f1071c: Beginning new tablet copy session on tablet 1cec7dfa2ba1466d9d5fc6e55baf280b from peer 3b41d87e7bfb419f8aa7500100129b6c at {username='slave'} at 127.0.99.134:40097: session id = 3b41d87e7bfb419f8aa7500100129b6c-1cec7dfa2ba1466d9d5fc6e55baf280b
I20240426 09:50:42.915836 6902 tablet_copy_source_session.cc:205] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:42.916834 31333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:42.919653 31333 tablet_copy_client.cc:783] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:42.919818 31333 tablet_copy_client.cc:647] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:42.923835 31333 tablet_copy_client.cc:523] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:42.925171 31333 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap starting.
I20240426 09:50:42.929359 31333 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:42.929816 31333 tablet_bootstrap.cc:492] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: Bootstrap complete.
I20240426 09:50:42.929962 31333 ts_tablet_manager.cc:1367] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240426 09:50:42.930368 31333 raft_consensus.cc:348] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:42.930470 31333 raft_consensus.cc:729] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Initialized, Role: FOLLOWER
I20240426 09:50:42.930593 31333 consensus_queue.cc:260] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:42.930881 31333 ts_tablet_manager.cc:1398] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240426 09:50:42.931311 6902 tablet_copy_service.cc:342] P c244e02f3cf847c0a5b69e1059f1071c: Request end of tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-1cec7dfa2ba1466d9d5fc6e55baf280b received from {username='slave'} at 127.0.99.134:40097
I20240426 09:50:42.931402 6902 tablet_copy_service.cc:433] P c244e02f3cf847c0a5b69e1059f1071c: ending tablet copy session 3b41d87e7bfb419f8aa7500100129b6c-1cec7dfa2ba1466d9d5fc6e55baf280b on tablet 1cec7dfa2ba1466d9d5fc6e55baf280b with peer 3b41d87e7bfb419f8aa7500100129b6c
I20240426 09:50:42.961165 31369 ts_tablet_manager.cc:902] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: Initiating tablet copy from peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135)
I20240426 09:50:42.961977 31369 tablet_copy_client.cc:308] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:37135
I20240426 09:50:42.963689 7033 tablet_copy_service.cc:140] P 0a0bfe28bcc0435ebe9a54a9d6416899: Received BeginTabletCopySession request for tablet 7799bbe0befb41d0a155ff51f41c23ad from peer 558c4ecd0ece460d9591aa7a3c799d19 ({username='slave'} at 127.0.99.136:44679)
I20240426 09:50:42.963802 7033 tablet_copy_service.cc:161] P 0a0bfe28bcc0435ebe9a54a9d6416899: Beginning new tablet copy session on tablet 7799bbe0befb41d0a155ff51f41c23ad from peer 558c4ecd0ece460d9591aa7a3c799d19 at {username='slave'} at 127.0.99.136:44679: session id = 558c4ecd0ece460d9591aa7a3c799d19-7799bbe0befb41d0a155ff51f41c23ad
I20240426 09:50:42.964771 7033 tablet_copy_source_session.cc:205] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 09:50:42.965649 31369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7799bbe0befb41d0a155ff51f41c23ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:42.970952 31369 tablet_copy_client.cc:783] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 0 data blocks...
I20240426 09:50:42.971139 31369 tablet_copy_client.cc:647] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Starting download of 1 WAL segments...
I20240426 09:50:42.976702 31369 tablet_copy_client.cc:523] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 09:50:42.979153 31369 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap starting.
I20240426 09:50:42.983525 31369 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 09:50:42.983809 31369 tablet_bootstrap.cc:492] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: Bootstrap complete.
I20240426 09:50:42.983975 31369 ts_tablet_manager.cc:1367] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20240426 09:50:42.984409 31369 raft_consensus.cc:348] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:42.984546 31369 raft_consensus.cc:729] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558c4ecd0ece460d9591aa7a3c799d19, State: Initialized, Role: FOLLOWER
I20240426 09:50:42.984664 31369 consensus_queue.cc:260] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:42.984933 31369 ts_tablet_manager.cc:1398] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240426 09:50:42.987668 7033 tablet_copy_service.cc:342] P 0a0bfe28bcc0435ebe9a54a9d6416899: Request end of tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-7799bbe0befb41d0a155ff51f41c23ad received from {username='slave'} at 127.0.99.136:44679
I20240426 09:50:42.987751 7033 tablet_copy_service.cc:433] P 0a0bfe28bcc0435ebe9a54a9d6416899: ending tablet copy session 558c4ecd0ece460d9591aa7a3c799d19-7799bbe0befb41d0a155ff51f41c23ad on tablet 7799bbe0befb41d0a155ff51f41c23ad with peer 558c4ecd0ece460d9591aa7a3c799d19
I20240426 09:50:43.072926 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:43.073478 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:43.075222 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:43.084960 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:43.092969 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:43.100802 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:43.104444 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:43.116005 9549 raft_consensus.cc:1203] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:43.128953 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:43.291078 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:43.293723 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:43.301038 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:43.302680 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:43.304754 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:43.306406 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:43.320119 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:43.340381 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:43.357113 7143 raft_consensus.cc:1203] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:43.418692 9417 raft_consensus.cc:1203] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:43.516209 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:43.516201 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:43.516930 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:43.521358 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:43.529202 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:43.536190 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:43.545022 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:43.555048 9396 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:43.573765 9672 raft_consensus.cc:1203] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240426 09:50:43.621831 7013 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "14752ab4ee9c4389b9d7afbfe1f5534c"
dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:36622
I20240426 09:50:43.622001 7013 raft_consensus.cc:595] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:43.754804 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:43.763386 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:43.767123 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:43.768944 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:43.769032 6992 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240426 09:50:43.776475 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:43.793954 31488 raft_consensus.cc:979] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: : Instructing follower 3b41d87e7bfb419f8aa7500100129b6c to start an election
I20240426 09:50:43.794102 31488 raft_consensus.cc:1067] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Signalling peer 3b41d87e7bfb419f8aa7500100129b6c to start an election
I20240426 09:50:43.794684 9417 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14752ab4ee9c4389b9d7afbfe1f5534c"
dest_uuid: "3b41d87e7bfb419f8aa7500100129b6c"
from {username='slave'} at 127.0.99.130:58979
I20240426 09:50:43.794844 9417 raft_consensus.cc:482] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:43.794899 9417 raft_consensus.cc:3037] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:43.796075 9417 raft_consensus.cc:504] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:43.797030 9417 leader_election.cc:288] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 2 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055), b8e863bc0edf4940bbf3e9f9b2bbb3e5 (127.0.99.135:36979)
I20240426 09:50:43.800156 7274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14752ab4ee9c4389b9d7afbfe1f5534c" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
I20240426 09:50:43.800276 7274 raft_consensus.cc:3037] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:43.801326 7274 raft_consensus.cc:2454] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 2.
I20240426 09:50:43.802423 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14752ab4ee9c4389b9d7afbfe1f5534c" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:50:43.802534 7013 raft_consensus.cc:3032] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:43.802568 7013 raft_consensus.cc:729] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:50:43.802708 7013 consensus_queue.cc:260] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:43.802922 7013 raft_consensus.cc:3037] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:43.803915 7013 raft_consensus.cc:2454] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 2.
I20240426 09:50:43.804464 9352 leader_election.cc:302] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 3b41d87e7bfb419f8aa7500100129b6c, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:50:43.804633 31446 raft_consensus.cc:2781] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:43.804700 31446 raft_consensus.cc:686] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 2 LEADER]: Becoming Leader. State: Replica: 3b41d87e7bfb419f8aa7500100129b6c, State: Running, Role: LEADER
I20240426 09:50:43.804823 31446 consensus_queue.cc:237] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:43.808022 9548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14752ab4ee9c4389b9d7afbfe1f5534c" candidate_uuid: "3b41d87e7bfb419f8aa7500100129b6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5"
I20240426 09:50:43.808166 9548 raft_consensus.cc:3037] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:43.809154 9548 raft_consensus.cc:2454] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b41d87e7bfb419f8aa7500100129b6c in term 2.
I20240426 09:50:43.809163 6731 catalog_manager.cc:5484] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c reported cstate change: term changed from 1 to 2, leader changed from 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) to 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134). New cstate: current_term: 2 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:43.810271 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:43.827252 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:44.209611 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:44.209815 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:44.210093 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:44.211097 7274 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:44.211468 31446 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:44.221958 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:44.228121 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:44.234339 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:44.235952 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:44.260744 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:44.321717 6880 consensus_queue.cc:237] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } }
I20240426 09:50:44.323081 9286 raft_consensus.cc:1261] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:44.323124 9416 raft_consensus.cc:1261] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:44.323573 30899 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:44.323870 31185 consensus_queue.cc:1035] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:44.325538 31185 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:44.326356 9286 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:44.327908 9416 raft_consensus.cc:2932] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:44.329905 6736 catalog_manager.cc:5484] T 1cec7dfa2ba1466d9d5fc6e55baf280b P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 4 to 5, VOTER 1aaf8befba984e199e96f53f46b50441 (127.0.99.131) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } }
I20240426 09:50:44.333765 7123 tablet_service.cc:1514] Processing DeleteTablet for tablet 1cec7dfa2ba1466d9d5fc6e55baf280b with delete_type TABLET_DATA_TOMBSTONED (TS 1aaf8befba984e199e96f53f46b50441 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52544
I20240426 09:50:44.334151 31628 tablet_replica.cc:330] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441: stopping tablet replica
I20240426 09:50:44.334314 31628 raft_consensus.cc:2229] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:44.334475 31628 raft_consensus.cc:2258] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:44.335004 31628 ts_tablet_manager.cc:1875] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:44.336581 9548 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:44.336972 31446 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:44.338138 31628 ts_tablet_manager.cc:1888] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:44.338335 31628 log.cc:1196] T 1cec7dfa2ba1466d9d5fc6e55baf280b P 1aaf8befba984e199e96f53f46b50441: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/wals/1cec7dfa2ba1466d9d5fc6e55baf280b
I20240426 09:50:44.339023 6724 catalog_manager.cc:4830] TS 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): tablet 1cec7dfa2ba1466d9d5fc6e55baf280b (table rebalancer_extra_table_56 [id=fd32695ea93a4402a52c5e6ed9e733d3]) successfully deleted
I20240426 09:50:44.339650 7013 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:44.340328 30887 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:44.472231 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:44.476310 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:44.478964 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:44.489485 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:44.489586 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:44.501978 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:44.503603 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:44.511499 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:44.588114 6882 consensus_queue.cc:237] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } }
I20240426 09:50:44.589504 7143 raft_consensus.cc:1261] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:44.589587 9285 raft_consensus.cc:1261] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Refusing update from remote peer c244e02f3cf847c0a5b69e1059f1071c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240426 09:50:44.589870 30899 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:44.590054 31185 consensus_queue.cc:1035] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:44.591660 30899 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:44.591987 7143 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:44.595105 6737 catalog_manager.cc:5484] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P c244e02f3cf847c0a5b69e1059f1071c reported cstate change: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New cstate: current_term: 1 leader_uuid: "c244e02f3cf847c0a5b69e1059f1071c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:44.595899 9285 raft_consensus.cc:2932] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P a692602e1a2e4794877e6fb92a7b9962 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "c244e02f3cf847c0a5b69e1059f1071c" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41571 } } peers { permanent_uuid: "a692602e1a2e4794877e6fb92a7b9962" member_type: VOTER last_known_addr { host: "127.0.99.133" port: 40545 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } }
I20240426 09:50:44.598666 9396 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 with delete_type TABLET_DATA_TOMBSTONED (TS 3b41d87e7bfb419f8aa7500100129b6c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:34916
I20240426 09:50:44.603574 31701 tablet_replica.cc:330] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c: stopping tablet replica
I20240426 09:50:44.603740 31701 raft_consensus.cc:2229] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:44.603902 31701 raft_consensus.cc:2258] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:44.604336 31701 ts_tablet_manager.cc:1875] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:44.606623 31701 ts_tablet_manager.cc:1888] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240426 09:50:44.606694 31701 log.cc:1196] T 2e78db4ae5cd4c40a3ed0478f8e9dee6 P 3b41d87e7bfb419f8aa7500100129b6c: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/wals/2e78db4ae5cd4c40a3ed0478f8e9dee6
I20240426 09:50:44.607255 6724 catalog_manager.cc:4830] TS 3b41d87e7bfb419f8aa7500100129b6c (127.0.99.134:37203): tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6 (table rebalancer_extra_table_58 [id=352cbb4ef14842a3b1255196c4cc33f1]) successfully deleted
I20240426 09:50:44.743270 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:44.745150 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:44.745404 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:44.753546 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:44.753790 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:44.760025 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:44.763859 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:44.772758 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:44.828330 7013 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7799bbe0befb41d0a155ff51f41c23ad"
dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:36658
I20240426 09:50:44.828500 7013 raft_consensus.cc:595] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Received request to transfer leadership
I20240426 09:50:44.959365 6992 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240426 09:50:44.966617 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:44.972523 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:44.973845 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:44.986876 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:44.994935 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:45.002167 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:45.017024 7254 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:45.080637 30711 raft_consensus.cc:979] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: : Instructing follower e3394a4be9ea4def8aa39b333c82c5ff to start an election
I20240426 09:50:45.080761 30711 raft_consensus.cc:1067] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Signalling peer e3394a4be9ea4def8aa39b333c82c5ff to start an election
I20240426 09:50:45.081322 7274 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7799bbe0befb41d0a155ff51f41c23ad"
dest_uuid: "e3394a4be9ea4def8aa39b333c82c5ff"
from {username='slave'} at 127.0.99.130:36325
I20240426 09:50:45.081473 7274 raft_consensus.cc:482] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:50:45.081537 7274 raft_consensus.cc:3037] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:45.082742 7274 raft_consensus.cc:504] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:45.083192 7274 leader_election.cc:288] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 2 election: Requested vote from peers 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135), 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021), 558c4ecd0ece460d9591aa7a3c799d19 (127.0.99.136:45059)
I20240426 09:50:45.086570 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7799bbe0befb41d0a155ff51f41c23ad" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "558c4ecd0ece460d9591aa7a3c799d19"
I20240426 09:50:45.086688 9672 raft_consensus.cc:3037] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:45.087003 7013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7799bbe0befb41d0a155ff51f41c23ad" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899"
I20240426 09:50:45.087126 7013 raft_consensus.cc:3032] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:50:45.087179 7013 raft_consensus.cc:729] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a0bfe28bcc0435ebe9a54a9d6416899, State: Running, Role: LEADER
I20240426 09:50:45.087337 7013 consensus_queue.cc:260] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:45.087550 7013 raft_consensus.cc:3037] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:45.087805 9672 raft_consensus.cc:2454] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 2.
I20240426 09:50:45.088042 7143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7799bbe0befb41d0a155ff51f41c23ad" candidate_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1aaf8befba984e199e96f53f46b50441"
I20240426 09:50:45.088146 7143 raft_consensus.cc:3037] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:50:45.088487 7013 raft_consensus.cc:2454] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 2.
I20240426 09:50:45.088824 7208 leader_election.cc:302] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a0bfe28bcc0435ebe9a54a9d6416899, 558c4ecd0ece460d9591aa7a3c799d19, e3394a4be9ea4def8aa39b333c82c5ff; no voters:
I20240426 09:50:45.089018 31809 raft_consensus.cc:2781] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:50:45.089128 31809 raft_consensus.cc:686] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 2 LEADER]: Becoming Leader. State: Replica: e3394a4be9ea4def8aa39b333c82c5ff, State: Running, Role: LEADER
I20240426 09:50:45.089118 7143 raft_consensus.cc:2454] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3394a4be9ea4def8aa39b333c82c5ff in term 2.
I20240426 09:50:45.089283 31809 consensus_queue.cc:237] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:45.091857 6736 catalog_manager.cc:5484] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: term changed from 1 to 2, leader changed from 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) to e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132). New cstate: current_term: 2 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:45.199715 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:45.206815 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:45.212482 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:45.212833 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:45.217065 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:45.225000 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:45.238988 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:45.257521 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:45.311182 9417 consensus_queue.cc:237] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } }
I20240426 09:50:45.312420 9546 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:45.312654 7274 raft_consensus.cc:1261] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Refusing update from remote peer 3b41d87e7bfb419f8aa7500100129b6c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:45.313019 31446 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:45.313242 31631 consensus_queue.cc:1035] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:45.314901 31881 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P 3b41d87e7bfb419f8aa7500100129b6c [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:45.315243 9548 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P b8e863bc0edf4940bbf3e9f9b2bbb3e5 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:45.315397 7274 raft_consensus.cc:2932] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:45.319911 6733 catalog_manager.cc:5484] T 14752ab4ee9c4389b9d7afbfe1f5534c P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 4 to 6, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "3b41d87e7bfb419f8aa7500100129b6c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3b41d87e7bfb419f8aa7500100129b6c" member_type: VOTER last_known_addr { host: "127.0.99.134" port: 37203 } } peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "b8e863bc0edf4940bbf3e9f9b2bbb3e5" member_type: VOTER last_known_addr { host: "127.0.99.135" port: 36979 } } }
I20240426 09:50:45.322562 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet 14752ab4ee9c4389b9d7afbfe1f5534c with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47654
I20240426 09:50:45.323035 31882 tablet_replica.cc:330] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:50:45.323194 31882 raft_consensus.cc:2229] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:45.323309 31882 raft_consensus.cc:2258] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:45.323797 31882 ts_tablet_manager.cc:1875] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:45.325798 31882 ts_tablet_manager.cc:1888] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:50:45.325868 31882 log.cc:1196] T 14752ab4ee9c4389b9d7afbfe1f5534c P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/14752ab4ee9c4389b9d7afbfe1f5534c
I20240426 09:50:45.326393 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 14752ab4ee9c4389b9d7afbfe1f5534c (table rebalancer_extra_table_62 [id=31457c26c8784127bf5491b3dcbd5dfe]) successfully deleted
I20240426 09:50:45.516492 7143 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:45.516872 31601 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:45.629582 7013 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:45.630031 31601 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0bfe28bcc0435ebe9a54a9d6416899" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:45.641377 9672 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:50:45.641786 31809 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240426 09:50:45.673976 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:45.681692 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:45.681914 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:45.701303 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:45.702418 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:45.708027 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:45.712579 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:45.713879 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:45.770301 7275 consensus_queue.cc:237] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } }
I20240426 09:50:45.771502 7142 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:45.771497 9672 raft_consensus.cc:1261] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Refusing update from remote peer e3394a4be9ea4def8aa39b333c82c5ff: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240426 09:50:45.771936 31809 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:45.772068 31808 consensus_queue.cc:1035] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240426 09:50:45.773923 31809 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:45.774283 9672 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P 558c4ecd0ece460d9591aa7a3c799d19 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:45.774623 7142 raft_consensus.cc:2932] T 7799bbe0befb41d0a155ff51f41c23ad P 1aaf8befba984e199e96f53f46b50441 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:45.778282 6732 catalog_manager.cc:5484] T 7799bbe0befb41d0a155ff51f41c23ad P e3394a4be9ea4def8aa39b333c82c5ff reported cstate change: config changed from index 4 to 6, VOTER 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130) evicted. New cstate: current_term: 2 leader_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e3394a4be9ea4def8aa39b333c82c5ff" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 38055 } } peers { permanent_uuid: "1aaf8befba984e199e96f53f46b50441" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 39021 } } peers { permanent_uuid: "558c4ecd0ece460d9591aa7a3c799d19" member_type: VOTER last_known_addr { host: "127.0.99.136" port: 45059 } } }
I20240426 09:50:45.781106 6992 tablet_service.cc:1514] Processing DeleteTablet for tablet 7799bbe0befb41d0a155ff51f41c23ad with delete_type TABLET_DATA_TOMBSTONED (TS 0a0bfe28bcc0435ebe9a54a9d6416899 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47654
I20240426 09:50:45.781343 31882 tablet_replica.cc:330] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: stopping tablet replica
I20240426 09:50:45.781486 31882 raft_consensus.cc:2229] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 09:50:45.781606 31882 raft_consensus.cc:2258] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 09:50:45.782274 31882 ts_tablet_manager.cc:1875] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 09:50:45.784462 31882 ts_tablet_manager.cc:1888] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240426 09:50:45.784513 31882 log.cc:1196] T 7799bbe0befb41d0a155ff51f41c23ad P 0a0bfe28bcc0435ebe9a54a9d6416899: Deleting WAL directory at /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/wals/7799bbe0befb41d0a155ff51f41c23ad
I20240426 09:50:45.785116 6724 catalog_manager.cc:4830] TS 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): tablet 7799bbe0befb41d0a155ff51f41c23ad (table rebalancer_extra_table_68 [id=34f4748289bf437aa40f4b6d60c44a7e]) successfully deleted
I20240426 09:50:45.980446 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:45.980561 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:45.997097 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:45.999927 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:46.003505 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:46.011468 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:46.028296 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:46.046068 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:46.269652 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:46.281191 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:46.288795 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:46.293877 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:46.294080 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:46.295895 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:46.296478 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:46.318956 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:46.711927 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:46.713559 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:46.716234 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:46.723670 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:46.726842 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:46.730122 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:46.732676 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:46.763773 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:47.013541 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:47.021194 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:47.029767 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:47.040467 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:47.041085 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:47.042518 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:47.044287 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:47.044607 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:47.318085 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:47.320905 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:47.324801 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:47.331521 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:47.339215 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:47.339481 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:47.349253 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:47.376633 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:47.769773 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:47.771000 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:47.774979 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:47.779994 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:47.782691 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:47.782747 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:47.783564 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:47.798296 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:48.081015 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:48.085444 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:48.087643 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:48.095976 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:48.102368 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:48.108940 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:48.127558 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:48.127568 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:48.406643 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:48.408386 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:48.412329 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:48.420007 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:48.424283 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:48.424713 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:48.431329 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:48.450947 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:48.710810 7123 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:48.712446 9264 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240426 09:50:48.720008 6992 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240426 09:50:48.721738 6862 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240426 09:50:48.722447 7254 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240426 09:50:48.740406 9396 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240426 09:50:48.755333 9660 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240426 09:50:48.762632 9527 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+--------------------+---------
27345b30edb745aa91af53b9ce01d7da | 127.0.99.190:43685 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:37513 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0a0bfe28bcc0435ebe9a54a9d6416899 | 127.0.99.130:37135 | HEALTHY | <none> | 23 | 0
1aaf8befba984e199e96f53f46b50441 | 127.0.99.131:39021 | HEALTHY | <none> | 17 | 0
3b41d87e7bfb419f8aa7500100129b6c | 127.0.99.134:37203 | HEALTHY | <none> | 16 | 0
558c4ecd0ece460d9591aa7a3c799d19 | 127.0.99.136:45059 | HEALTHY | <none> | 20 | 0
a692602e1a2e4794877e6fb92a7b9962 | 127.0.99.133:40545 | HEALTHY | <none> | 14 | 0
b8e863bc0edf4940bbf3e9f9b2bbb3e5 | 127.0.99.135:36979 | HEALTHY | <none> | 20 | 0
c244e02f3cf847c0a5b69e1059f1071c | 127.0.99.129:41571 | HEALTHY | <none> | 15 | 0
e3394a4be9ea4def8aa39b333c82c5ff | 127.0.99.132:38055 | HEALTHY | <none> | 17 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.0.99.129 | experimental | 127.0.99.129:41571
local_ip_for_outbound_sockets | 127.0.99.130 | experimental | 127.0.99.130:37135
local_ip_for_outbound_sockets | 127.0.99.131 | experimental | 127.0.99.131:39021
local_ip_for_outbound_sockets | 127.0.99.132 | experimental | 127.0.99.132:38055
local_ip_for_outbound_sockets | 127.0.99.133 | experimental | 127.0.99.133:40545
local_ip_for_outbound_sockets | 127.0.99.134 | experimental | 127.0.99.134:37203
local_ip_for_outbound_sockets | 127.0.99.135 | experimental | 127.0.99.135:36979
local_ip_for_outbound_sockets | 127.0.99.136 | experimental | 127.0.99.136:45059
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.0.99.129:41571
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.0.99.130:37135
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.0.99.131:39021
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.0.99.132:38055
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.0.99.133:40545
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.0.99.134:37203
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.0.99.135:36979
server_dump_info_path | /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.0.99.136:45059
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.0.99.148:37513 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------------------+----+---------+---------------+---------+------------+------------------+-------------
rebalance_test_table_1 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_11 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_13 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_15 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_17 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_19 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_5 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_7 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_9 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_dynamic_table | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalancer_extra_table_0 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_10 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_12 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_14 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_16 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_18 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_2 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_20 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_22 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_24 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_26 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_28 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_30 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_32 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_34 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_36 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_38 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_4 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_40 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_42 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_44 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_46 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_48 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_50 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_52 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_54 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_56 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_58 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_6 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_60 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_62 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_64 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_66 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_68 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_70 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_72 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalancer_extra_table_8 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 53
First Quartile | 53
Median | 53
Third Quartile | 54
Maximum | 54
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 48
Tablets | 142
Replicas | 426
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240426 09:50:48.830389 398 log_verifier.cc:126] Checking tablet 01d9726ca8b641178824a7fa5977bbdf
I20240426 09:50:48.855093 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 01d9726ca8b641178824a7fa5977bbdf
I20240426 09:50:48.855157 398 log_verifier.cc:126] Checking tablet 03489b207ed042bba1cab08aff3afd67
I20240426 09:50:48.880661 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 03489b207ed042bba1cab08aff3afd67
I20240426 09:50:48.880721 398 log_verifier.cc:126] Checking tablet 0362b7d68d9048bca47d96e948563d03
I20240426 09:50:48.910202 398 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 0362b7d68d9048bca47d96e948563d03
I20240426 09:50:48.910272 398 log_verifier.cc:126] Checking tablet 08fa514a62a84c0e8068bfe23d175773
I20240426 09:50:48.934573 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 08fa514a62a84c0e8068bfe23d175773
I20240426 09:50:48.934695 398 log_verifier.cc:126] Checking tablet 0c04f49b228c4c6d9bd6a222d8f287cc
I20240426 09:50:48.960538 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 0c04f49b228c4c6d9bd6a222d8f287cc
I20240426 09:50:48.960618 398 log_verifier.cc:126] Checking tablet 0d04a7ec2c62437b879de204b5c885e9
I20240426 09:50:48.986997 398 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 0d04a7ec2c62437b879de204b5c885e9
I20240426 09:50:48.987094 398 log_verifier.cc:126] Checking tablet 0f67d94463ce46708246808da0b100e9
I20240426 09:50:49.017390 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 0f67d94463ce46708246808da0b100e9
I20240426 09:50:49.017455 398 log_verifier.cc:126] Checking tablet 10118b6d060f4999a2cdfdaa1b8de197
I20240426 09:50:49.043093 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 10118b6d060f4999a2cdfdaa1b8de197
I20240426 09:50:49.043170 398 log_verifier.cc:126] Checking tablet 11604d6436a946eba63a36c28e1b0290
I20240426 09:50:49.071089 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 11604d6436a946eba63a36c28e1b0290
I20240426 09:50:49.071156 398 log_verifier.cc:126] Checking tablet 11a10ff62d224cf49fcb71d11b029537
I20240426 09:50:49.103135 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 11a10ff62d224cf49fcb71d11b029537
I20240426 09:50:49.103199 398 log_verifier.cc:126] Checking tablet 14752ab4ee9c4389b9d7afbfe1f5534c
I20240426 09:50:49.129460 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 14752ab4ee9c4389b9d7afbfe1f5534c
I20240426 09:50:49.129527 398 log_verifier.cc:126] Checking tablet 158666afa8f14791addcd6be64e6c562
I20240426 09:50:49.155953 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 158666afa8f14791addcd6be64e6c562
I20240426 09:50:49.156018 398 log_verifier.cc:126] Checking tablet 1a99c72632ab49a5b7b2c4a5b305b119
I20240426 09:50:49.182940 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1a99c72632ab49a5b7b2c4a5b305b119
I20240426 09:50:49.183005 398 log_verifier.cc:126] Checking tablet 1c2fd1740d544f5d98741aecb64cdfc7
I20240426 09:50:49.183137 398 log_verifier.cc:177] Verified matching terms for 0 ops in tablet 1c2fd1740d544f5d98741aecb64cdfc7
I20240426 09:50:49.183164 398 log_verifier.cc:126] Checking tablet 1cec7dfa2ba1466d9d5fc6e55baf280b
I20240426 09:50:49.210180 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 1cec7dfa2ba1466d9d5fc6e55baf280b
I20240426 09:50:49.210251 398 log_verifier.cc:126] Checking tablet 1d720846739e49f888da92befe06a309
I20240426 09:50:49.236217 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1d720846739e49f888da92befe06a309
I20240426 09:50:49.236315 398 log_verifier.cc:126] Checking tablet 1f1b9c46feae4f659dc2eeb8619c5b4e
I20240426 09:50:49.261130 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 1f1b9c46feae4f659dc2eeb8619c5b4e
I20240426 09:50:49.261194 398 log_verifier.cc:126] Checking tablet 1fa5a98645834e2886ff45b8b28c1c91
I20240426 09:50:49.289670 398 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 1fa5a98645834e2886ff45b8b28c1c91
I20240426 09:50:49.289747 398 log_verifier.cc:126] Checking tablet 22930f3d5b514c9884359f6c939c53e3
I20240426 09:50:49.314810 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 22930f3d5b514c9884359f6c939c53e3
I20240426 09:50:49.314870 398 log_verifier.cc:126] Checking tablet 2770cd77ebb642e3a52183a3871e33fa
I20240426 09:50:49.341416 398 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 2770cd77ebb642e3a52183a3871e33fa
I20240426 09:50:49.341487 398 log_verifier.cc:126] Checking tablet 279519370ee640fd9bbd3028008f1d5d
I20240426 09:50:49.368556 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 279519370ee640fd9bbd3028008f1d5d
I20240426 09:50:49.368643 398 log_verifier.cc:126] Checking tablet 295e6e95d86c47c88a09e47448c93d2e
I20240426 09:50:49.400676 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 295e6e95d86c47c88a09e47448c93d2e
I20240426 09:50:49.400753 398 log_verifier.cc:126] Checking tablet 2c41aab70e4a4b4e86cf095575cd8805
I20240426 09:50:49.427924 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 2c41aab70e4a4b4e86cf095575cd8805
I20240426 09:50:49.428026 398 log_verifier.cc:126] Checking tablet 2e249a2325ef4bbf9be46998f505e3fc
I20240426 09:50:49.453320 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 2e249a2325ef4bbf9be46998f505e3fc
I20240426 09:50:49.453402 398 log_verifier.cc:126] Checking tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6
I20240426 09:50:49.479760 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 2e78db4ae5cd4c40a3ed0478f8e9dee6
I20240426 09:50:49.479825 398 log_verifier.cc:126] Checking tablet 3086b0bb58174f4db7c29db9a74dae88
I20240426 09:50:49.504580 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 3086b0bb58174f4db7c29db9a74dae88
I20240426 09:50:49.504662 398 log_verifier.cc:126] Checking tablet 325e1d125617456f877204314d9e7bb9
I20240426 09:50:49.534397 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 325e1d125617456f877204314d9e7bb9
I20240426 09:50:49.534468 398 log_verifier.cc:126] Checking tablet 348e90f5c63e4eda89b87fc103896302
I20240426 09:50:49.561089 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 348e90f5c63e4eda89b87fc103896302
I20240426 09:50:49.561170 398 log_verifier.cc:126] Checking tablet 360aa06557974c5ea635a32245c0903e
I20240426 09:50:49.587966 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 360aa06557974c5ea635a32245c0903e
I20240426 09:50:49.588063 398 log_verifier.cc:126] Checking tablet 369c5672f4924725a7da4019241dc772
I20240426 09:50:49.615911 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 369c5672f4924725a7da4019241dc772
I20240426 09:50:49.615984 398 log_verifier.cc:126] Checking tablet 3bd97a192183433aa48531ab7885e73e
I20240426 09:50:49.645884 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 3bd97a192183433aa48531ab7885e73e
I20240426 09:50:49.645953 398 log_verifier.cc:126] Checking tablet 3c05f4899b1d441f8553f2dc970f6ed3
I20240426 09:50:49.674002 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 3c05f4899b1d441f8553f2dc970f6ed3
I20240426 09:50:49.674067 398 log_verifier.cc:126] Checking tablet 3d5f900639e2494b97e13f3703536f42
I20240426 09:50:49.699903 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 3d5f900639e2494b97e13f3703536f42
I20240426 09:50:49.699966 398 log_verifier.cc:126] Checking tablet 3f8803aa38a646ebbdf5018ab7f65845
I20240426 09:50:49.725037 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 3f8803aa38a646ebbdf5018ab7f65845
I20240426 09:50:49.725106 398 log_verifier.cc:126] Checking tablet 4189112964fb425aac074352dd873415
I20240426 09:50:49.755198 398 log_verifier.cc:177] Verified matching terms for 8 ops in tablet 4189112964fb425aac074352dd873415
I20240426 09:50:49.755281 398 log_verifier.cc:126] Checking tablet 4613f57999fa4a6baf5ccfd102f5c73b
I20240426 09:50:49.781127 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 4613f57999fa4a6baf5ccfd102f5c73b
I20240426 09:50:49.781234 398 log_verifier.cc:126] Checking tablet 4d49693f3ade40f2a9d5a0029dd7508b
I20240426 09:50:49.808593 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 4d49693f3ade40f2a9d5a0029dd7508b
I20240426 09:50:49.808656 398 log_verifier.cc:126] Checking tablet 4da482df496340a0a41f7a4f3fed4021
I20240426 09:50:49.834450 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 4da482df496340a0a41f7a4f3fed4021
I20240426 09:50:49.834512 398 log_verifier.cc:126] Checking tablet 4e344f55669d4b16a8d3d5ef274b87fe
I20240426 09:50:49.859127 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 4e344f55669d4b16a8d3d5ef274b87fe
I20240426 09:50:49.859206 398 log_verifier.cc:126] Checking tablet 4e3624733f3444778b8656b3289d57dc
I20240426 09:50:49.885361 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 4e3624733f3444778b8656b3289d57dc
I20240426 09:50:49.885427 398 log_verifier.cc:126] Checking tablet 4fb0f5d2d55a45599c1b09c5684522e6
I20240426 09:50:49.916949 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 4fb0f5d2d55a45599c1b09c5684522e6
I20240426 09:50:49.917019 398 log_verifier.cc:126] Checking tablet 5040b4c862a84a5786857fb0f53849ca
I20240426 09:50:49.942807 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 5040b4c862a84a5786857fb0f53849ca
I20240426 09:50:49.942925 398 log_verifier.cc:126] Checking tablet 536858c15b894b2f82efef38b8f51594
I20240426 09:50:49.975181 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 536858c15b894b2f82efef38b8f51594
I20240426 09:50:49.975243 398 log_verifier.cc:126] Checking tablet 53765d0ffcc24bb98aadc0cd0fe44af3
I20240426 09:50:50.002363 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 53765d0ffcc24bb98aadc0cd0fe44af3
I20240426 09:50:50.002444 398 log_verifier.cc:126] Checking tablet 538173a16ff94e70bc8d25b8f1e7c84e
I20240426 09:50:50.028829 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 538173a16ff94e70bc8d25b8f1e7c84e
I20240426 09:50:50.028901 398 log_verifier.cc:126] Checking tablet 5951b29e98cc4ff3a459dec654580ec0
I20240426 09:50:50.059144 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 5951b29e98cc4ff3a459dec654580ec0
I20240426 09:50:50.059218 398 log_verifier.cc:126] Checking tablet 5b5e68d9b3ce49d3aa398e85e139ddbf
I20240426 09:50:50.086036 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 5b5e68d9b3ce49d3aa398e85e139ddbf
I20240426 09:50:50.086103 398 log_verifier.cc:126] Checking tablet 5b66a5a6cf3348ffbaa56679985e9aa2
I20240426 09:50:50.112308 398 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 5b66a5a6cf3348ffbaa56679985e9aa2
I20240426 09:50:50.112372 398 log_verifier.cc:126] Checking tablet 5ba744a156204d9e81db6ea113945203
I20240426 09:50:50.143090 398 log_verifier.cc:177] Verified matching terms for 8 ops in tablet 5ba744a156204d9e81db6ea113945203
I20240426 09:50:50.143168 398 log_verifier.cc:126] Checking tablet 5e010e7f54b54b04afe4e3b1df0c739f
I20240426 09:50:50.168908 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 5e010e7f54b54b04afe4e3b1df0c739f
I20240426 09:50:50.168972 398 log_verifier.cc:126] Checking tablet 5f5418f85c2b49ce89f82331689e1b30
I20240426 09:50:50.195427 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 5f5418f85c2b49ce89f82331689e1b30
I20240426 09:50:50.195501 398 log_verifier.cc:126] Checking tablet 5fe43b0765d9401a88c8e9a028ab3ef4
I20240426 09:50:50.225699 398 log_verifier.cc:177] Verified matching terms for 8 ops in tablet 5fe43b0765d9401a88c8e9a028ab3ef4
I20240426 09:50:50.225773 398 log_verifier.cc:126] Checking tablet 6112d2390c994fefb74b8e7f75d57d17
I20240426 09:50:50.252147 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 6112d2390c994fefb74b8e7f75d57d17
I20240426 09:50:50.252218 398 log_verifier.cc:126] Checking tablet 61bc946b265243198d1cb16247c43b7f
I20240426 09:50:50.277303 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 61bc946b265243198d1cb16247c43b7f
I20240426 09:50:50.277369 398 log_verifier.cc:126] Checking tablet 635293adcd7b45858935a65e4ba58f24
I20240426 09:50:50.301862 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 635293adcd7b45858935a65e4ba58f24
I20240426 09:50:50.301944 398 log_verifier.cc:126] Checking tablet 638b15a2c9ea42c3b86a985ccae550a7
I20240426 09:50:50.326752 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 638b15a2c9ea42c3b86a985ccae550a7
I20240426 09:50:50.326854 398 log_verifier.cc:126] Checking tablet 639d1d6189fb4d5f8a2d0aa0d39575ec
I20240426 09:50:50.357319 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 639d1d6189fb4d5f8a2d0aa0d39575ec
I20240426 09:50:50.357390 398 log_verifier.cc:126] Checking tablet 63f066721f3441299fe5dffd07c1afed
I20240426 09:50:50.386961 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 63f066721f3441299fe5dffd07c1afed
I20240426 09:50:50.387027 398 log_verifier.cc:126] Checking tablet 6404a6d30b534fe3a1748a2e486579fc
I20240426 09:50:50.412158 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 6404a6d30b534fe3a1748a2e486579fc
I20240426 09:50:50.412222 398 log_verifier.cc:126] Checking tablet 643fd2d359a14f8eaa8d8a0d0879ef61
I20240426 09:50:50.442953 398 log_verifier.cc:177] Verified matching terms for 8 ops in tablet 643fd2d359a14f8eaa8d8a0d0879ef61
I20240426 09:50:50.443044 398 log_verifier.cc:126] Checking tablet 67e3f93c0a2944e38b7542d1439c907e
I20240426 09:50:50.469584 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 67e3f93c0a2944e38b7542d1439c907e
I20240426 09:50:50.469668 398 log_verifier.cc:126] Checking tablet 6983bc4d17d44f998a749f7ec95bb305
I20240426 09:50:50.495568 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 6983bc4d17d44f998a749f7ec95bb305
I20240426 09:50:50.495632 398 log_verifier.cc:126] Checking tablet 6a1b8658361f4c4a834e8c886909fe95
I20240426 09:50:50.521167 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 6a1b8658361f4c4a834e8c886909fe95
I20240426 09:50:50.521258 398 log_verifier.cc:126] Checking tablet 6c2a1fd5edcf4323b61d7d8a4ea610f2
I20240426 09:50:50.545661 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 6c2a1fd5edcf4323b61d7d8a4ea610f2
I20240426 09:50:50.545733 398 log_verifier.cc:126] Checking tablet 6dc287ff7ea640a38537c74e9ff156f7
I20240426 09:50:50.545899 398 log_verifier.cc:177] Verified matching terms for 0 ops in tablet 6dc287ff7ea640a38537c74e9ff156f7
I20240426 09:50:50.545922 398 log_verifier.cc:126] Checking tablet 6dce6f1e165c492ab23022527bac580a
I20240426 09:50:50.572383 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 6dce6f1e165c492ab23022527bac580a
I20240426 09:50:50.572458 398 log_verifier.cc:126] Checking tablet 6e2f793f7e7a4ff596379c1cfe9d251b
I20240426 09:50:50.597689 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 6e2f793f7e7a4ff596379c1cfe9d251b
I20240426 09:50:50.597750 398 log_verifier.cc:126] Checking tablet 6e3fb0b19aa34a53831c27c4d7490007
I20240426 09:50:50.622215 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 6e3fb0b19aa34a53831c27c4d7490007
I20240426 09:50:50.622280 398 log_verifier.cc:126] Checking tablet 6f6f0978441e4ce69c6e40805e495fd0
I20240426 09:50:50.646895 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 6f6f0978441e4ce69c6e40805e495fd0
I20240426 09:50:50.646976 398 log_verifier.cc:126] Checking tablet 722306427b7b46b4b0942193c821d990
I20240426 09:50:50.672873 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 722306427b7b46b4b0942193c821d990
I20240426 09:50:50.672942 398 log_verifier.cc:126] Checking tablet 753f199b5cb2426ba90e8ad003efcb32
I20240426 09:50:50.698318 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 753f199b5cb2426ba90e8ad003efcb32
I20240426 09:50:50.698390 398 log_verifier.cc:126] Checking tablet 757210fd3040442e8195888c70666948
I20240426 09:50:50.723641 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 757210fd3040442e8195888c70666948
I20240426 09:50:50.723703 398 log_verifier.cc:126] Checking tablet 7799bbe0befb41d0a155ff51f41c23ad
I20240426 09:50:50.749511 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 7799bbe0befb41d0a155ff51f41c23ad
I20240426 09:50:50.749595 398 log_verifier.cc:126] Checking tablet 7e130e7733534aedb221d7772a2d29ce
I20240426 09:50:50.777999 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 7e130e7733534aedb221d7772a2d29ce
I20240426 09:50:50.778076 398 log_verifier.cc:126] Checking tablet 7e494cf4a72945148ad4afa658ff1b03
I20240426 09:50:50.803668 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 7e494cf4a72945148ad4afa658ff1b03
I20240426 09:50:50.803756 398 log_verifier.cc:126] Checking tablet 7eecd02c620e4b1eb17548a94e6a9ac1
I20240426 09:50:50.829334 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 7eecd02c620e4b1eb17548a94e6a9ac1
I20240426 09:50:50.829407 398 log_verifier.cc:126] Checking tablet 838bef0a35c046e39b465f2ea743350f
I20240426 09:50:50.855217 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 838bef0a35c046e39b465f2ea743350f
I20240426 09:50:50.855281 398 log_verifier.cc:126] Checking tablet 845d1058274a41d49b6ac67571cb5439
I20240426 09:50:50.881239 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 845d1058274a41d49b6ac67571cb5439
I20240426 09:50:50.881314 398 log_verifier.cc:126] Checking tablet 852fad44c59f42dd9ce66b84819a9296
I20240426 09:50:50.909583 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 852fad44c59f42dd9ce66b84819a9296
I20240426 09:50:50.909673 398 log_verifier.cc:126] Checking tablet 87d356a7ac32469c824af0970730214f
I20240426 09:50:50.935886 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 87d356a7ac32469c824af0970730214f
I20240426 09:50:50.935950 398 log_verifier.cc:126] Checking tablet 8c623acd7cc9480faec733d2fd109117
I20240426 09:50:50.961817 398 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 8c623acd7cc9480faec733d2fd109117
I20240426 09:50:50.961887 398 log_verifier.cc:126] Checking tablet 8cb2e6c9a044463aa58878e06fc3fff2
I20240426 09:50:50.988761 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 8cb2e6c9a044463aa58878e06fc3fff2
I20240426 09:50:50.988845 398 log_verifier.cc:126] Checking tablet 8e306113f9874277a60468d9248d45ce
I20240426 09:50:51.016564 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 8e306113f9874277a60468d9248d45ce
I20240426 09:50:51.016664 398 log_verifier.cc:126] Checking tablet 91a15bb5ed1541ca906b8603d9249690
I20240426 09:50:51.043689 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 91a15bb5ed1541ca906b8603d9249690
I20240426 09:50:51.043766 398 log_verifier.cc:126] Checking tablet 937837e55579473087c0d0fcd8bcb001
I20240426 09:50:51.069413 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 937837e55579473087c0d0fcd8bcb001
I20240426 09:50:51.069486 398 log_verifier.cc:126] Checking tablet 948e1df8d52f42cebcd289fff377d6fb
I20240426 09:50:51.098450 398 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 948e1df8d52f42cebcd289fff377d6fb
I20240426 09:50:51.098515 398 log_verifier.cc:126] Checking tablet 956f89ba28c54473964cc877c26580f0
I20240426 09:50:51.127610 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 956f89ba28c54473964cc877c26580f0
I20240426 09:50:51.127673 398 log_verifier.cc:126] Checking tablet 957533c35bae4299b35b77ba9a465eb3
I20240426 09:50:51.152841 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 957533c35bae4299b35b77ba9a465eb3
I20240426 09:50:51.152904 398 log_verifier.cc:126] Checking tablet 96ba8066c09d403bb5299d25d6568a76
I20240426 09:50:51.178341 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 96ba8066c09d403bb5299d25d6568a76
I20240426 09:50:51.178412 398 log_verifier.cc:126] Checking tablet 981b71e5e8b94db7b909ec0b7564152a
I20240426 09:50:51.205575 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 981b71e5e8b94db7b909ec0b7564152a
I20240426 09:50:51.205672 398 log_verifier.cc:126] Checking tablet 988b6b1007634a8980398e4f68dd4c4d
I20240426 09:50:51.230296 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 988b6b1007634a8980398e4f68dd4c4d
I20240426 09:50:51.230383 398 log_verifier.cc:126] Checking tablet 9a9c1e00452f4f32b31e8b336892df49
I20240426 09:50:51.257979 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet 9a9c1e00452f4f32b31e8b336892df49
I20240426 09:50:51.258042 398 log_verifier.cc:126] Checking tablet 9c94c820ab65420a8d3f885ccc9f8884
I20240426 09:50:51.288758 398 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 9c94c820ab65420a8d3f885ccc9f8884
I20240426 09:50:51.288833 398 log_verifier.cc:126] Checking tablet a31b0f75c2514916a94205ce0289629e
I20240426 09:50:51.315135 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet a31b0f75c2514916a94205ce0289629e
I20240426 09:50:51.315202 398 log_verifier.cc:126] Checking tablet a42302cab4524e7a85defe4da40be268
I20240426 09:50:51.343487 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet a42302cab4524e7a85defe4da40be268
I20240426 09:50:51.343564 398 log_verifier.cc:126] Checking tablet a8912e37d7e941a791dd77560e2b3b6b
I20240426 09:50:51.374193 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet a8912e37d7e941a791dd77560e2b3b6b
I20240426 09:50:51.374275 398 log_verifier.cc:126] Checking tablet ab631877cfb646829d984aacd8a02795
I20240426 09:50:51.400684 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet ab631877cfb646829d984aacd8a02795
I20240426 09:50:51.400769 398 log_verifier.cc:126] Checking tablet ab6450f48a36483883f8914075c224a8
I20240426 09:50:51.427623 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet ab6450f48a36483883f8914075c224a8
I20240426 09:50:51.427691 398 log_verifier.cc:126] Checking tablet aca60cffb8cf41e3b0d9b64436c306d7
I20240426 09:50:51.456058 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet aca60cffb8cf41e3b0d9b64436c306d7
I20240426 09:50:51.456117 398 log_verifier.cc:126] Checking tablet aebef9b26fec446b8e71d4046768aedc
I20240426 09:50:51.481388 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet aebef9b26fec446b8e71d4046768aedc
I20240426 09:50:51.481456 398 log_verifier.cc:126] Checking tablet af4d2322360b43fdbd04aa430f41494e
I20240426 09:50:51.506309 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet af4d2322360b43fdbd04aa430f41494e
I20240426 09:50:51.506402 398 log_verifier.cc:126] Checking tablet b017fe28f5f241af8955642664afaaa8
I20240426 09:50:51.531718 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet b017fe28f5f241af8955642664afaaa8
I20240426 09:50:51.531795 398 log_verifier.cc:126] Checking tablet b08039d1ab5e4b108ae21a24c507c876
I20240426 09:50:51.557030 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet b08039d1ab5e4b108ae21a24c507c876
I20240426 09:50:51.557106 398 log_verifier.cc:126] Checking tablet b0a674c2acbe4f69a7b8fd30fa12b833
I20240426 09:50:51.588271 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet b0a674c2acbe4f69a7b8fd30fa12b833
I20240426 09:50:51.588361 398 log_verifier.cc:126] Checking tablet b2d42c52b7a94e32ab4cdad78ce884ee
I20240426 09:50:51.620467 398 log_verifier.cc:177] Verified matching terms for 7 ops in tablet b2d42c52b7a94e32ab4cdad78ce884ee
I20240426 09:50:51.620538 398 log_verifier.cc:126] Checking tablet b32cacd1e7354933928c427f9646cafb
I20240426 09:50:51.647688 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet b32cacd1e7354933928c427f9646cafb
I20240426 09:50:51.647766 398 log_verifier.cc:126] Checking tablet b5a4a67b5ab943b0a6299d74f2411190
I20240426 09:50:51.673259 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet b5a4a67b5ab943b0a6299d74f2411190
I20240426 09:50:51.673329 398 log_verifier.cc:126] Checking tablet b674ff3fd5284b02b7e5b3fc62145a0a
I20240426 09:50:51.698141 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet b674ff3fd5284b02b7e5b3fc62145a0a
I20240426 09:50:51.698202 398 log_verifier.cc:126] Checking tablet b925f8cd837b4e5f8fd3f6f608d3ad31
I20240426 09:50:51.722995 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet b925f8cd837b4e5f8fd3f6f608d3ad31
I20240426 09:50:51.723073 398 log_verifier.cc:126] Checking tablet b967d83e93d843398e93077285865b1b
I20240426 09:50:51.748562 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet b967d83e93d843398e93077285865b1b
I20240426 09:50:51.748625 398 log_verifier.cc:126] Checking tablet bbdcd6105a2647738257638e1e1096c3
I20240426 09:50:51.774274 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet bbdcd6105a2647738257638e1e1096c3
I20240426 09:50:51.774349 398 log_verifier.cc:126] Checking tablet bd077eafe82d4531a6850e6686a7e566
I20240426 09:50:51.804718 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet bd077eafe82d4531a6850e6686a7e566
I20240426 09:50:51.804805 398 log_verifier.cc:126] Checking tablet be115ad80a4b42edbc6b2c630e54aedc
I20240426 09:50:51.835062 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet be115ad80a4b42edbc6b2c630e54aedc
I20240426 09:50:51.835122 398 log_verifier.cc:126] Checking tablet c0a2aef96c3845b7bb6357b9914fe7f8
I20240426 09:50:51.860311 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet c0a2aef96c3845b7bb6357b9914fe7f8
I20240426 09:50:51.860379 398 log_verifier.cc:126] Checking tablet c10b7673a7bf4980add3d94c6284e57b
I20240426 09:50:51.886219 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet c10b7673a7bf4980add3d94c6284e57b
I20240426 09:50:51.886284 398 log_verifier.cc:126] Checking tablet c6956266cb444536945540ecdbaba28e
I20240426 09:50:51.912112 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet c6956266cb444536945540ecdbaba28e
I20240426 09:50:51.912175 398 log_verifier.cc:126] Checking tablet c6eb1eaad9ac4dc9aa94739e482ebd9c
I20240426 09:50:51.936576 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet c6eb1eaad9ac4dc9aa94739e482ebd9c
I20240426 09:50:51.936641 398 log_verifier.cc:126] Checking tablet ca0a2b121aba4659bf2d3a475a1e298b
I20240426 09:50:51.964190 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet ca0a2b121aba4659bf2d3a475a1e298b
I20240426 09:50:51.964275 398 log_verifier.cc:126] Checking tablet cd377032e7cd4b4998424f8685ad5b85
I20240426 09:50:51.996704 398 log_verifier.cc:177] Verified matching terms for 7 ops in tablet cd377032e7cd4b4998424f8685ad5b85
I20240426 09:50:51.996773 398 log_verifier.cc:126] Checking tablet cd67a40e0b834a83b41259ee4da3f3db
I20240426 09:50:52.021756 398 log_verifier.cc:177] Verified matching terms for 4 ops in tablet cd67a40e0b834a83b41259ee4da3f3db
I20240426 09:50:52.021827 398 log_verifier.cc:126] Checking tablet ce1e72dc91434c4ab63639fcc50da2ec
I20240426 09:50:52.051332 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet ce1e72dc91434c4ab63639fcc50da2ec
I20240426 09:50:52.051403 398 log_verifier.cc:126] Checking tablet ce43c4c9b2994813ad7ad38f2e1b1040
I20240426 09:50:52.077119 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet ce43c4c9b2994813ad7ad38f2e1b1040
I20240426 09:50:52.077183 398 log_verifier.cc:126] Checking tablet cfe16e4114924bda8c560d4926a63923
I20240426 09:50:52.103296 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet cfe16e4114924bda8c560d4926a63923
I20240426 09:50:52.103369 398 log_verifier.cc:126] Checking tablet d0779912d7a04b4c95dd1db6f9c4d6fd
I20240426 09:50:52.130108 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet d0779912d7a04b4c95dd1db6f9c4d6fd
I20240426 09:50:52.130187 398 log_verifier.cc:126] Checking tablet d1d91861708146eb94dcc2cdef1517eb
I20240426 09:50:52.158296 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet d1d91861708146eb94dcc2cdef1517eb
I20240426 09:50:52.158423 398 log_verifier.cc:126] Checking tablet d2a5223682934aa698dd23aa6eab941b
I20240426 09:50:52.185040 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet d2a5223682934aa698dd23aa6eab941b
I20240426 09:50:52.185117 398 log_verifier.cc:126] Checking tablet d30be76596534071a3ea9a90375099f6
I20240426 09:50:52.209960 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet d30be76596534071a3ea9a90375099f6
I20240426 09:50:52.210022 398 log_verifier.cc:126] Checking tablet d60546e3101f4cbd9bc137d73ed5dea8
I20240426 09:50:52.234831 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet d60546e3101f4cbd9bc137d73ed5dea8
I20240426 09:50:52.234905 398 log_verifier.cc:126] Checking tablet d6981747bd8c44d2bb56f3d5bb56dbb1
I20240426 09:50:52.260082 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet d6981747bd8c44d2bb56f3d5bb56dbb1
I20240426 09:50:52.260143 398 log_verifier.cc:126] Checking tablet d83c01e6dc6d4b5b9365243058f459a1
I20240426 09:50:52.286357 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet d83c01e6dc6d4b5b9365243058f459a1
I20240426 09:50:52.286438 398 log_verifier.cc:126] Checking tablet da33173a2b0c45ba95cebdf61c6a7499
I20240426 09:50:52.311476 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet da33173a2b0c45ba95cebdf61c6a7499
I20240426 09:50:52.311553 398 log_verifier.cc:126] Checking tablet df622ccf230d49a58ce29308f3234690
I20240426 09:50:52.336786 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet df622ccf230d49a58ce29308f3234690
I20240426 09:50:52.336863 398 log_verifier.cc:126] Checking tablet e110e59d534c4bbd9fbceae078bb8286
I20240426 09:50:52.363042 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet e110e59d534c4bbd9fbceae078bb8286
I20240426 09:50:52.363104 398 log_verifier.cc:126] Checking tablet e3114093f8f24864b7184f0bd91fb401
I20240426 09:50:52.389060 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet e3114093f8f24864b7184f0bd91fb401
I20240426 09:50:52.389122 398 log_verifier.cc:126] Checking tablet e5baaf8ce5a247e2975f8d6327349a26
I20240426 09:50:52.417939 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet e5baaf8ce5a247e2975f8d6327349a26
I20240426 09:50:52.418002 398 log_verifier.cc:126] Checking tablet e64e5170d6bc4154b5ec869d535051c3
I20240426 09:50:52.442575 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet e64e5170d6bc4154b5ec869d535051c3
I20240426 09:50:52.442668 398 log_verifier.cc:126] Checking tablet e6cb4534d05c49ed859b4abe61097ac8
I20240426 09:50:52.468298 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet e6cb4534d05c49ed859b4abe61097ac8
I20240426 09:50:52.468406 398 log_verifier.cc:126] Checking tablet e856ff2c8ffb4bd4b44c694047c8a8fa
I20240426 09:50:52.493927 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet e856ff2c8ffb4bd4b44c694047c8a8fa
I20240426 09:50:52.494004 398 log_verifier.cc:126] Checking tablet ecd7063af4854e17bb2211e31033a89b
I20240426 09:50:52.520275 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet ecd7063af4854e17bb2211e31033a89b
I20240426 09:50:52.520361 398 log_verifier.cc:126] Checking tablet efa472b0759c4cbbb7db13dc3fbe0b50
I20240426 09:50:52.548185 398 log_verifier.cc:177] Verified matching terms for 6 ops in tablet efa472b0759c4cbbb7db13dc3fbe0b50
I20240426 09:50:52.548265 398 log_verifier.cc:126] Checking tablet f1bb1cdfa9684023ad017c0e36cb44b1
I20240426 09:50:52.580633 398 log_verifier.cc:177] Verified matching terms for 3 ops in tablet f1bb1cdfa9684023ad017c0e36cb44b1
I20240426 09:50:52.580718 398 log_verifier.cc:126] Checking tablet f834d8a08b1341869fd644ff400fceae
I20240426 09:50:52.622437 398 log_verifier.cc:177] Verified matching terms for 5 ops in tablet f834d8a08b1341869fd644ff400fceae
I20240426 09:50:52.622525 398 log_verifier.cc:126] Checking tablet fb3c46383718465783af89bb0ad4896c
I20240426 09:50:52.655618 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet fb3c46383718465783af89bb0ad4896c
I20240426 09:50:52.655709 398 log_verifier.cc:126] Checking tablet feb385fea62d4d6fa355b49461c7a05a
I20240426 09:50:52.688516 398 log_verifier.cc:177] Verified matching terms for 1 ops in tablet feb385fea62d4d6fa355b49461c7a05a
I20240426 09:50:52.689225 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 6800
I20240426 09:50:52.702795 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 6931
W20240426 09:50:52.713951 7077 connection.cc:535] client connection to 127.0.99.130:37135 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240426 09:50:52.714108 7077 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240426 09:50:52.714296 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 7062
W20240426 09:50:52.716820 9484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.129:41571: connect: Connection refused (error 111)
W20240426 09:50:52.717717 9484 consensus_peers.cc:485] T 8cb2e6c9a044463aa58878e06fc3fff2 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.129:41571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240426 09:50:52.718197 9350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.130:37135: connect: Connection refused (error 111)
W20240426 09:50:52.718992 9350 consensus_peers.cc:485] T 2770cd77ebb642e3a52183a3871e33fa P 3b41d87e7bfb419f8aa7500100129b6c -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.130:37135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240426 09:50:52.723798 9221 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.129:41571: connect: Connection refused (error 111)
W20240426 09:50:52.724491 9221 consensus_peers.cc:485] T 4189112964fb425aac074352dd873415 P a692602e1a2e4794877e6fb92a7b9962 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.129:41571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240426 09:50:52.727277 9614 connection.cc:535] client connection to 127.0.99.131:39021 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240426 09:50:52.727432 9614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240426 09:50:52.729604 32484 negotiation.cc:337] Failed RPC negotiation. Trace:
0426 09:50:52.728096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.99.131:39021 (local address 127.0.99.136:43427)
0426 09:50:52.728302 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0426 09:50:52.728314 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0426 09:50:52.728448 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0426 09:50:52.729028 (+ 580us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.99.131:39021: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
I20240426 09:50:52.729620 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 7193
W20240426 09:50:52.735947 9221 consensus_peers.cc:485] T aebef9b26fec446b8e71d4046768aedc P a692602e1a2e4794877e6fb92a7b9962 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.129:41571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240426 09:50:52.736747 9614 consensus_peers.cc:485] T ca0a2b121aba4659bf2d3a475a1e298b P 558c4ecd0ece460d9591aa7a3c799d19 -> Peer 1aaf8befba984e199e96f53f46b50441 (127.0.99.131:39021): Couldn't send request to peer 1aaf8befba984e199e96f53f46b50441. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.131:39021: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20240426 09:50:52.737875 9484 consensus_peers.cc:485] T 96ba8066c09d403bb5299d25d6568a76 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.129:41571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240426 09:50:52.740259 9350 consensus_peers.cc:485] T 158666afa8f14791addcd6be64e6c562 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer 0a0bfe28bcc0435ebe9a54a9d6416899 (127.0.99.130:37135): Couldn't send request to peer 0a0bfe28bcc0435ebe9a54a9d6416899. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.130:37135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:52.741276 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 9204
W20240426 09:50:52.743877 9484 consensus_peers.cc:485] T d6981747bd8c44d2bb56f3d5bb56dbb1 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer c244e02f3cf847c0a5b69e1059f1071c (127.0.99.129:41571): Couldn't send request to peer c244e02f3cf847c0a5b69e1059f1071c. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.129:41571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240426 09:50:52.745055 9352 consensus_peers.cc:485] T b08039d1ab5e4b108ae21a24c507c876 P 3b41d87e7bfb419f8aa7500100129b6c -> Peer e3394a4be9ea4def8aa39b333c82c5ff (127.0.99.132:38055): Couldn't send request to peer e3394a4be9ea4def8aa39b333c82c5ff. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.132:38055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:52.750756 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 9334
W20240426 09:50:52.757897 9483 consensus_peers.cc:485] T 369c5672f4924725a7da4019241dc772 P b8e863bc0edf4940bbf3e9f9b2bbb3e5 -> Peer a692602e1a2e4794877e6fb92a7b9962 (127.0.99.133:40545): Couldn't send request to peer a692602e1a2e4794877e6fb92a7b9962. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.133:40545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240426 09:50:52.759797 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 9466
I20240426 09:50:52.768494 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 9598
I20240426 09:50:52.776633 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 6709
2024-04-26T09:50:52Z chronyd exiting
[ OK ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/0 (100462 ms)
[----------] 1 test from DDLDuringRebalancingTest (100462 ms total)
[----------] 1 test from RebalancingDuringElectionStormTest
[ RUN ] RebalancingDuringElectionStormTest.RoundRobin/0
I20240426 09:50:53.210734 398 test_util.cc:268] Using random seed: -689590167
I20240426 09:50:53.211745 398 ts_itest-base.cc:112] Starting cluster with:
I20240426 09:50:53.211796 398 ts_itest-base.cc:113] --------------
I20240426 09:50:53.211813 398 ts_itest-base.cc:114] 8 tablet servers
I20240426 09:50:53.211830 398 ts_itest-base.cc:115] 3 replicas per TS
I20240426 09:50:53.211849 398 ts_itest-base.cc:116] --------------
2024-04-26T09:50:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-04-26T09:50:53Z Disabled control of system clock
I20240426 09:50:53.220284 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:37003
--webserver_interface=127.0.99.190
--webserver_port=0
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.99.190:37003
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240426 09:50:53.324985 32492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:53.325215 32492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:50:53.325284 32492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:53.328228 32492 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:50:53.328303 32492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240426 09:50:53.328344 32492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:50:53.328382 32492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240426 09:50:53.328423 32492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240426 09:50:53.332037 32492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.0.99.190:37003
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.99.190:37003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.0.99.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.32492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:50:53.332957 32492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:50:53.334069 32492 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:50:53.342351 32498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:53.343132 32497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:53.344179 32492 server_base.cc:1034] running on GCE node
W20240426 09:50:53.344970 32500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:53.345480 32492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:50:53.346443 32492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:50:53.347594 32492 hybrid_clock.cc:648] HybridClock initialized: now 1714125053347580 us; error 30 us; skew 500 ppm
I20240426 09:50:53.353727 32492 webserver.cc:460] Webserver started at http://127.0.99.190:46033/ using document root <none> and password file <none>
I20240426 09:50:53.354296 32492 fs_manager.cc:359] Metadata directory not provided
I20240426 09:50:53.354357 32492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:50:53.354534 32492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:50:53.356317 32492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "e8fb13ed4998488baa1a093ce3d0e9fa"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.356793 32492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "e8fb13ed4998488baa1a093ce3d0e9fa"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.360076 32492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240426 09:50:53.362315 32506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.363354 32492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240426 09:50:53.363473 32492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
uuid: "e8fb13ed4998488baa1a093ce3d0e9fa"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.363554 32492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:50:53.374809 32492 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:50:53.375442 32492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:50:53.375597 32492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:50:53.382040 32492 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:37003
I20240426 09:50:53.382047 32558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:37003 every 8 connection(s)
I20240426 09:50:53.383020 32492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240426 09:50:53.385474 32559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:53.385609 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 32492
I20240426 09:50:53.385749 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240426 09:50:53.391712 32559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa: Bootstrap starting.
I20240426 09:50:53.394054 32559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:53.394896 32559 log.cc:824] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa: Log is configured to *not* fsync() on all Append() calls
I20240426 09:50:53.396466 32559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa: No bootstrap required, opened a new log
I20240426 09:50:53.398927 32559 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb13ed4998488baa1a093ce3d0e9fa" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 37003 } }
I20240426 09:50:53.399124 32559 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:53.399184 32559 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8fb13ed4998488baa1a093ce3d0e9fa, State: Initialized, Role: FOLLOWER
I20240426 09:50:53.399587 32559 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb13ed4998488baa1a093ce3d0e9fa" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 37003 } }
I20240426 09:50:53.399703 32559 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 09:50:53.399760 32559 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 09:50:53.399864 32559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:53.400529 32559 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb13ed4998488baa1a093ce3d0e9fa" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 37003 } }
I20240426 09:50:53.400820 32559 leader_election.cc:302] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8fb13ed4998488baa1a093ce3d0e9fa; no voters:
I20240426 09:50:53.401067 32559 leader_election.cc:288] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20240426 09:50:53.401165 32564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:53.401365 32564 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [term 1 LEADER]: Becoming Leader. State: Replica: e8fb13ed4998488baa1a093ce3d0e9fa, State: Running, Role: LEADER
I20240426 09:50:53.401662 32564 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb13ed4998488baa1a093ce3d0e9fa" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 37003 } }
I20240426 09:50:53.402079 32559 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [sys.catalog]: configured and running, proceeding with master startup.
I20240426 09:50:53.403188 32565 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8fb13ed4998488baa1a093ce3d0e9fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb13ed4998488baa1a093ce3d0e9fa" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 37003 } } }
I20240426 09:50:53.403192 32566 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8fb13ed4998488baa1a093ce3d0e9fa. Latest consensus state: current_term: 1 leader_uuid: "e8fb13ed4998488baa1a093ce3d0e9fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fb13ed4998488baa1a093ce3d0e9fa" member_type: VOTER last_known_addr { host: "127.0.99.190" port: 37003 } } }
I20240426 09:50:53.403384 32566 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [sys.catalog]: This master's current role is: LEADER
I20240426 09:50:53.403343 32565 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa [sys.catalog]: This master's current role is: LEADER
I20240426 09:50:53.403713 32573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 09:50:53.406459 32573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 09:50:53.411540 32573 catalog_manager.cc:1347] Generated new cluster ID: 9d54d877574d40d1a77b77fc36401b09
I20240426 09:50:53.411608 32573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 09:50:53.423615 32573 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 09:50:53.424787 32573 catalog_manager.cc:1504] Loading token signing keys...
I20240426 09:50:53.433367 32573 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e8fb13ed4998488baa1a093ce3d0e9fa: Generated new TSK 0
I20240426 09:50:53.434077 32573 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 09:50:53.449251 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.129:0
--local_ip_for_outbound_sockets=127.0.99.129
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:50:53.567028 32583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:53.567246 32583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:50:53.567315 32583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:53.570305 32583 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:50:53.570374 32583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:50:53.570464 32583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.129
I20240426 09:50:53.573843 32583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.0.99.129
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.32583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.129
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:50:53.574671 32583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:50:53.575831 32583 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:50:53.584431 32588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:53.585301 32591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:53.586124 32589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:53.586957 32583 server_base.cc:1034] running on GCE node
I20240426 09:50:53.587306 32583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:50:53.587815 32583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:50:53.588961 32583 hybrid_clock.cc:648] HybridClock initialized: now 1714125053588950 us; error 25 us; skew 500 ppm
I20240426 09:50:53.590845 32583 webserver.cc:460] Webserver started at http://127.0.99.129:33231/ using document root <none> and password file <none>
I20240426 09:50:53.591370 32583 fs_manager.cc:359] Metadata directory not provided
I20240426 09:50:53.591424 32583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:50:53.591610 32583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:50:53.593284 32583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "342778fb25b446c68cdcd20f98bcd8b2"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.593736 32583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "342778fb25b446c68cdcd20f98bcd8b2"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.596967 32583 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240426 09:50:53.598976 32597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.599960 32583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:53.600061 32583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "342778fb25b446c68cdcd20f98bcd8b2"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.600137 32583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:50:53.627522 32583 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:50:53.628129 32583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:50:53.628284 32583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:50:53.628862 32583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:50:53.629738 32583 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:50:53.629783 32583 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.629822 32583 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:50:53.629843 32583 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.637861 32583 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:36999
I20240426 09:50:53.637894 32710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:36999 every 8 connection(s)
I20240426 09:50:53.638862 32583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240426 09:50:53.646070 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 32583
I20240426 09:50:53.646179 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240426 09:50:53.647028 32711 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:50:53.647248 32711 heartbeater.cc:464] Registering TS with master...
I20240426 09:50:53.647763 32711 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:50:53.648177 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.130:0
--local_ip_for_outbound_sockets=127.0.99.130
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:50:53.649091 32523 ts_manager.cc:194] Registered new tserver with Master: 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:50:53.650738 32523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.129:60959
W20240426 09:50:53.751885 32714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:53.752106 32714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:50:53.752162 32714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:53.755051 32714 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:50:53.755110 32714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:50:53.755174 32714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.130
I20240426 09:50:53.758544 32714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.0.99.130
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.32714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.130
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:50:53.759421 32714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:50:53.760470 32714 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:50:53.769126 32719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:53.769910 32720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:53.770884 32722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:53.771471 32714 server_base.cc:1034] running on GCE node
I20240426 09:50:53.771853 32714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:50:53.772394 32714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:50:53.773566 32714 hybrid_clock.cc:648] HybridClock initialized: now 1714125053773554 us; error 26 us; skew 500 ppm
I20240426 09:50:53.775490 32714 webserver.cc:460] Webserver started at http://127.0.99.130:34941/ using document root <none> and password file <none>
I20240426 09:50:53.776018 32714 fs_manager.cc:359] Metadata directory not provided
I20240426 09:50:53.776078 32714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:50:53.776247 32714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:50:53.777844 32714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "07593a4e1c0242a1b52bb96338b63c8c"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.778297 32714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "07593a4e1c0242a1b52bb96338b63c8c"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.781673 32714 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240426 09:50:53.783785 32728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.784746 32714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:50:53.784847 32714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "07593a4e1c0242a1b52bb96338b63c8c"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.784929 32714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:50:53.796844 32714 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:50:53.797574 32714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:50:53.797747 32714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:50:53.798254 32714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:50:53.799146 32714 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:50:53.799196 32714 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.799248 32714 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:50:53.799276 32714 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.807919 32714 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:37271
I20240426 09:50:53.807942 373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:37271 every 8 connection(s)
I20240426 09:50:53.808799 32714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240426 09:50:53.814261 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 32714
I20240426 09:50:53.814371 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240426 09:50:53.816515 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.131:0
--local_ip_for_outbound_sockets=127.0.99.131
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:50:53.817116 374 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:50:53.817340 374 heartbeater.cc:464] Registering TS with master...
I20240426 09:50:53.817865 374 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:50:53.818819 32523 ts_manager.cc:194] Registered new tserver with Master: 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:50:53.819330 32523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.130:59895
W20240426 09:50:53.920665 378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:53.920893 378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:50:53.920961 378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:53.923858 378 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:50:53.923933 378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:50:53.924016 378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.131
I20240426 09:50:53.927340 378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.0.99.131
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.131
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:50:53.928170 378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:50:53.929256 378 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:50:53.937984 384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:53.938683 383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:53.939549 386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:53.940379 378 server_base.cc:1034] running on GCE node
I20240426 09:50:53.940799 378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:50:53.941332 378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:50:53.942477 378 hybrid_clock.cc:648] HybridClock initialized: now 1714125053942462 us; error 32 us; skew 500 ppm
I20240426 09:50:53.944460 378 webserver.cc:460] Webserver started at http://127.0.99.131:46831/ using document root <none> and password file <none>
I20240426 09:50:53.944998 378 fs_manager.cc:359] Metadata directory not provided
I20240426 09:50:53.945058 378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:50:53.945233 378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:50:53.946949 378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.947407 378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.950557 378 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240426 09:50:53.952683 392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.953641 378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240426 09:50:53.953754 378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
format_stamp: "Formatted at 2024-04-26 09:50:53 on dist-test-slave-1zqj"
I20240426 09:50:53.953830 378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:50:53.967082 378 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:50:53.967736 378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:50:53.967887 378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:50:53.968380 378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:50:53.969236 378 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:50:53.969283 378 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.969337 378 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:50:53.969368 378 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:53.977494 378 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:46223
I20240426 09:50:53.977523 510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:46223 every 8 connection(s)
I20240426 09:50:53.978384 378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240426 09:50:53.982322 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 378
I20240426 09:50:53.982407 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240426 09:50:53.984387 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.132:0
--local_ip_for_outbound_sockets=127.0.99.132
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:50:53.987037 511 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:50:53.987282 511 heartbeater.cc:464] Registering TS with master...
I20240426 09:50:53.987823 511 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:50:53.988859 32523 ts_manager.cc:194] Registered new tserver with Master: 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:50:53.989457 32523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.131:37025
W20240426 09:50:54.089088 514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:54.089318 514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:50:54.089391 514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:54.092286 514 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:50:54.092353 514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:50:54.092450 514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.132
I20240426 09:50:54.096068 514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.0.99.132
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.132
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:50:54.096992 514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:50:54.098050 514 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:50:54.106820 520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.107753 519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.108614 522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:54.109498 514 server_base.cc:1034] running on GCE node
I20240426 09:50:54.109891 514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:50:54.110421 514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:50:54.111582 514 hybrid_clock.cc:648] HybridClock initialized: now 1714125054111556 us; error 38 us; skew 500 ppm
I20240426 09:50:54.113515 514 webserver.cc:460] Webserver started at http://127.0.99.132:39991/ using document root <none> and password file <none>
I20240426 09:50:54.114051 514 fs_manager.cc:359] Metadata directory not provided
I20240426 09:50:54.114106 514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:50:54.114279 514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:50:54.115969 514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "f9655c9ff2e14edfbcd798d88bca0623"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.116410 514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "f9655c9ff2e14edfbcd798d88bca0623"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.119669 514 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240426 09:50:54.121697 528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.122613 514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:54.122721 514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "f9655c9ff2e14edfbcd798d88bca0623"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.122838 514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:50:54.137194 514 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:50:54.137902 514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:50:54.138054 514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:50:54.138712 514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:50:54.140141 514 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:50:54.140201 514 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.140262 514 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:50:54.140286 514 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.152318 514 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.132:40661
I20240426 09:50:54.152359 641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.132:40661 every 8 connection(s)
I20240426 09:50:54.153653 514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240426 09:50:54.160856 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 514
I20240426 09:50:54.160982 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240426 09:50:54.163579 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:0
--local_ip_for_outbound_sockets=127.0.99.133
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:50:54.166910 642 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:50:54.167158 642 heartbeater.cc:464] Registering TS with master...
I20240426 09:50:54.167606 642 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:50:54.168630 32523 ts_manager.cc:194] Registered new tserver with Master: f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:50:54.169170 32523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.132:42945
W20240426 09:50:54.268235 645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:54.268455 645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:50:54.268534 645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:54.271378 645 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:50:54.271437 645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:50:54.271497 645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240426 09:50:54.275008 645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:50:54.275794 645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:50:54.276806 645 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:50:54.285108 653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.285938 651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.286726 650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:54.287591 645 server_base.cc:1034] running on GCE node
I20240426 09:50:54.287955 645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:50:54.288462 645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:50:54.289613 645 hybrid_clock.cc:648] HybridClock initialized: now 1714125054289591 us; error 45 us; skew 500 ppm
I20240426 09:50:54.291427 645 webserver.cc:460] Webserver started at http://127.0.99.133:36651/ using document root <none> and password file <none>
I20240426 09:50:54.291954 645 fs_manager.cc:359] Metadata directory not provided
I20240426 09:50:54.292028 645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:50:54.292192 645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:50:54.294099 645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "c252379311f0425ca38566356d6b41c2"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.295903 645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "c252379311f0425ca38566356d6b41c2"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.299242 645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240426 09:50:54.301249 659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.302187 645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:54.302289 645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "c252379311f0425ca38566356d6b41c2"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.302368 645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:50:54.316628 645 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:50:54.317355 645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:50:54.317528 645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:50:54.318264 645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:50:54.319506 645 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:50:54.319577 645 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.319648 645 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:50:54.319681 645 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.329030 645 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:42497
I20240426 09:50:54.329066 772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:42497 every 8 connection(s)
I20240426 09:50:54.329942 645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240426 09:50:54.338274 773 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:50:54.338488 773 heartbeater.cc:464] Registering TS with master...
I20240426 09:50:54.338985 773 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:50:54.339576 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 645
I20240426 09:50:54.339681 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240426 09:50:54.339962 32523 ts_manager.cc:194] Registered new tserver with Master: c252379311f0425ca38566356d6b41c2 (127.0.99.133:42497)
I20240426 09:50:54.340533 32523 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:50713
I20240426 09:50:54.341784 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:0
--local_ip_for_outbound_sockets=127.0.99.134
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:50:54.448829 776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:54.449043 776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:50:54.449115 776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:54.452037 776 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:50:54.452103 776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:50:54.452219 776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240426 09:50:54.455772 776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:50:54.456627 776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:50:54.457710 776 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:50:54.466023 781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.466849 782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.467672 784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:54.468446 776 server_base.cc:1034] running on GCE node
I20240426 09:50:54.468815 776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:50:54.469336 776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:50:54.470489 776 hybrid_clock.cc:648] HybridClock initialized: now 1714125054470481 us; error 26 us; skew 500 ppm
I20240426 09:50:54.472328 776 webserver.cc:460] Webserver started at http://127.0.99.134:37693/ using document root <none> and password file <none>
I20240426 09:50:54.472862 776 fs_manager.cc:359] Metadata directory not provided
I20240426 09:50:54.472919 776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:50:54.473084 776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:50:54.474681 776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "b22f98253a344ccd969920c43529f24e"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.475193 776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "b22f98253a344ccd969920c43529f24e"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.478260 776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240426 09:50:54.480326 790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.481266 776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:50:54.481375 776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "b22f98253a344ccd969920c43529f24e"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.481459 776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:50:54.514359 776 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:50:54.515067 776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:50:54.515226 776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:50:54.515762 776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:50:54.516779 776 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:50:54.516835 776 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.516896 776 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:50:54.516925 776 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.524662 776 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:35801
I20240426 09:50:54.524689 903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:35801 every 8 connection(s)
I20240426 09:50:54.525543 776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240426 09:50:54.527686 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 776
I20240426 09:50:54.527787 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240426 09:50:54.529942 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:0
--local_ip_for_outbound_sockets=127.0.99.135
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:50:54.534721 904 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:50:54.535032 904 heartbeater.cc:464] Registering TS with master...
I20240426 09:50:54.535619 904 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:50:54.536660 32522 ts_manager.cc:194] Registered new tserver with Master: b22f98253a344ccd969920c43529f24e (127.0.99.134:35801)
I20240426 09:50:54.537159 32522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:59931
W20240426 09:50:54.636734 907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:54.636950 907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:50:54.637008 907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:54.640045 907 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:50:54.640162 907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:50:54.640285 907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240426 09:50:54.643743 907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:50:54.644555 907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:50:54.645623 907 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240426 09:50:54.652971 32711 heartbeater.cc:503] Master 127.0.99.190:37003 was elected leader, sending a full tablet report...
W20240426 09:50:54.654114 912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.654973 913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.655817 915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:54.656697 907 server_base.cc:1034] running on GCE node
I20240426 09:50:54.657063 907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:50:54.657625 907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:50:54.658798 907 hybrid_clock.cc:648] HybridClock initialized: now 1714125054658781 us; error 28 us; skew 500 ppm
I20240426 09:50:54.660570 907 webserver.cc:460] Webserver started at http://127.0.99.135:32921/ using document root <none> and password file <none>
I20240426 09:50:54.661087 907 fs_manager.cc:359] Metadata directory not provided
I20240426 09:50:54.661146 907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:50:54.661311 907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:50:54.662955 907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "777b53878fab4bc885635b63dd63ae2e"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.663383 907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "777b53878fab4bc885635b63dd63ae2e"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.666383 907 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240426 09:50:54.668421 921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.669368 907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:50:54.669485 907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "777b53878fab4bc885635b63dd63ae2e"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.669566 907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:50:54.688186 907 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:50:54.688859 907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:50:54.689018 907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:50:54.689553 907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:50:54.690402 907 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:50:54.690451 907 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.690505 907 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:50:54.690532 907 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.699271 907 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:46491
I20240426 09:50:54.699301 1034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:46491 every 8 connection(s)
I20240426 09:50:54.700222 907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240426 09:50:54.706218 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 907
I20240426 09:50:54.706331 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240426 09:50:54.708487 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:0
--local_ip_for_outbound_sockets=127.0.99.136
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:50:54.709156 1035 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:50:54.709344 1035 heartbeater.cc:464] Registering TS with master...
I20240426 09:50:54.709764 1035 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:50:54.710788 32522 ts_manager.cc:194] Registered new tserver with Master: 777b53878fab4bc885635b63dd63ae2e (127.0.99.135:46491)
I20240426 09:50:54.711300 32522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:48465
W20240426 09:50:54.816179 1038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:54.816399 1038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:50:54.816468 1038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:54.819419 1038 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:50:54.819494 1038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:50:54.819583 1038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240426 09:50:54.821061 374 heartbeater.cc:503] Master 127.0.99.190:37003 was elected leader, sending a full tablet report...
I20240426 09:50:54.823123 1038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=0
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.1038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:50:54.823983 1038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:50:54.825045 1038 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:50:54.833535 1043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.834290 1046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:50:54.835147 1044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:50:54.836035 1038 server_base.cc:1034] running on GCE node
I20240426 09:50:54.836369 1038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:50:54.836894 1038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:50:54.838037 1038 hybrid_clock.cc:648] HybridClock initialized: now 1714125054838025 us; error 25 us; skew 500 ppm
I20240426 09:50:54.839934 1038 webserver.cc:460] Webserver started at http://127.0.99.136:34675/ using document root <none> and password file <none>
I20240426 09:50:54.840457 1038 fs_manager.cc:359] Metadata directory not provided
I20240426 09:50:54.840508 1038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:50:54.840674 1038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 09:50:54.842226 1038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "e87b815a8bac4e03ab414f74df93f02a"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.842660 1038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "e87b815a8bac4e03ab414f74df93f02a"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.845914 1038 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240426 09:50:54.847982 1052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.848963 1038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:50:54.849061 1038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e87b815a8bac4e03ab414f74df93f02a"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:50:54.849135 1038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:50:54.868899 1038 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:50:54.869491 1038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:50:54.869627 1038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:50:54.870133 1038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:50:54.870965 1038 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:50:54.871008 1038 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.871065 1038 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:50:54.871085 1038 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:50:54.879518 1038 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:37421
I20240426 09:50:54.879539 1165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:37421 every 8 connection(s)
I20240426 09:50:54.880438 1038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240426 09:50:54.884353 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 1038
I20240426 09:50:54.884450 398 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240426 09:50:54.889014 1166 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:50:54.889240 1166 heartbeater.cc:464] Registering TS with master...
I20240426 09:50:54.889700 1166 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:50:54.890601 32522 ts_manager.cc:194] Registered new tserver with Master: e87b815a8bac4e03ab414f74df93f02a (127.0.99.136:37421)
I20240426 09:50:54.891147 32522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:51235
I20240426 09:50:54.898864 398 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240426 09:50:54.908280 398 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240426 09:50:54.908356 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 645
I20240426 09:50:54.914058 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 776
I20240426 09:50:54.919422 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 907
I20240426 09:50:54.925081 398 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu with pid 1038
I20240426 09:50:54.991820 511 heartbeater.cc:503] Master 127.0.99.190:37003 was elected leader, sending a full tablet report...
I20240426 09:50:55.171293 642 heartbeater.cc:503] Master 127.0.99.190:37003 was elected leader, sending a full tablet report...
I20240426 09:50:58.682377 32522 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:50:58.702844 307 tablet_service.cc:1467] Processing CreateTablet for tablet e9771706a67b4a12b64cc43a991bbd27 (DEFAULT_TABLE table=rebalance_test_table_0 [id=83d7984fd8ea4c1a8ee6ce023822e26c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:58.703050 444 tablet_service.cc:1467] Processing CreateTablet for tablet 451f3108a6fb494da331dd6a6ec3e290 (DEFAULT_TABLE table=rebalance_test_table_0 [id=83d7984fd8ea4c1a8ee6ce023822e26c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:58.702705 308 tablet_service.cc:1467] Processing CreateTablet for tablet 451f3108a6fb494da331dd6a6ec3e290 (DEFAULT_TABLE table=rebalance_test_table_0 [id=83d7984fd8ea4c1a8ee6ce023822e26c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:58.702987 301 tablet_service.cc:1467] Processing CreateTablet for tablet b2ecfdfe7ac44636b5d4b620c05ed1c9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=83d7984fd8ea4c1a8ee6ce023822e26c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:58.703897 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451f3108a6fb494da331dd6a6ec3e290. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:58.703897 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451f3108a6fb494da331dd6a6ec3e290. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:58.704242 307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9771706a67b4a12b64cc43a991bbd27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:58.706127 576 tablet_service.cc:1467] Processing CreateTablet for tablet e9771706a67b4a12b64cc43a991bbd27 (DEFAULT_TABLE table=rebalance_test_table_0 [id=83d7984fd8ea4c1a8ee6ce023822e26c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:58.707016 559 tablet_service.cc:1467] Processing CreateTablet for tablet b2ecfdfe7ac44636b5d4b620c05ed1c9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=83d7984fd8ea4c1a8ee6ce023822e26c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:58.707438 559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2ecfdfe7ac44636b5d4b620c05ed1c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:58.710724 1186 tablet_bootstrap.cc:492] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:50:58.711856 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2ecfdfe7ac44636b5d4b620c05ed1c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:58.712960 1186 tablet_bootstrap.cc:654] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:58.713768 1186 log.cc:824] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b: Log is configured to *not* fsync() on all Append() calls
I20240426 09:50:58.702713 445 tablet_service.cc:1467] Processing CreateTablet for tablet b2ecfdfe7ac44636b5d4b620c05ed1c9 (DEFAULT_TABLE table=rebalance_test_table_0 [id=83d7984fd8ea4c1a8ee6ce023822e26c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:58.713728 32645 tablet_service.cc:1467] Processing CreateTablet for tablet 451f3108a6fb494da331dd6a6ec3e290 (DEFAULT_TABLE table=rebalance_test_table_0 [id=83d7984fd8ea4c1a8ee6ce023822e26c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:58.714208 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2ecfdfe7ac44636b5d4b620c05ed1c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:58.714449 1187 tablet_bootstrap.cc:492] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:50:58.714646 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451f3108a6fb494da331dd6a6ec3e290. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:58.715008 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9771706a67b4a12b64cc43a991bbd27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:58.715066 32644 tablet_service.cc:1467] Processing CreateTablet for tablet e9771706a67b4a12b64cc43a991bbd27 (DEFAULT_TABLE table=rebalance_test_table_0 [id=83d7984fd8ea4c1a8ee6ce023822e26c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:58.716543 1187 tablet_bootstrap.cc:654] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:58.717308 1187 log.cc:824] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c: Log is configured to *not* fsync() on all Append() calls
I20240426 09:50:58.718142 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9771706a67b4a12b64cc43a991bbd27. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:58.719116 1187 tablet_bootstrap.cc:492] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:50:58.719290 1187 ts_tablet_manager.cc:1367] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20240426 09:50:58.719367 1186 tablet_bootstrap.cc:492] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:50:58.719537 1186 ts_tablet_manager.cc:1367] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20240426 09:50:58.719890 1188 tablet_bootstrap.cc:492] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:50:58.722074 1188 tablet_bootstrap.cc:654] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:58.722910 1188 log.cc:824] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623: Log is configured to *not* fsync() on all Append() calls
I20240426 09:50:58.723155 1191 tablet_bootstrap.cc:492] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:50:58.723205 1186 raft_consensus.cc:348] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.723402 1186 raft_consensus.cc:374] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:58.723455 1186 raft_consensus.cc:729] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:50:58.723935 1186 consensus_queue.cc:260] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.723950 1187 raft_consensus.cc:348] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.724117 1187 raft_consensus.cc:374] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:58.724155 1187 raft_consensus.cc:729] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:50:58.724571 1187 consensus_queue.cc:260] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.724761 1186 ts_tablet_manager.cc:1398] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240426 09:50:58.724779 1188 tablet_bootstrap.cc:492] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:50:58.724982 1188 ts_tablet_manager.cc:1367] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240426 09:50:58.725070 1191 tablet_bootstrap.cc:654] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:58.725353 1187 ts_tablet_manager.cc:1398] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20240426 09:50:58.725677 1187 tablet_bootstrap.cc:492] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:50:58.725788 1191 log.cc:824] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2: Log is configured to *not* fsync() on all Append() calls
I20240426 09:50:58.726737 1187 tablet_bootstrap.cc:654] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:58.727355 1191 tablet_bootstrap.cc:492] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:50:58.727521 1191 ts_tablet_manager.cc:1367] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240426 09:50:58.727957 1186 tablet_bootstrap.cc:492] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:50:58.728484 1188 raft_consensus.cc:348] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.728621 1188 raft_consensus.cc:374] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:58.728650 1188 raft_consensus.cc:729] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:50:58.728855 1186 tablet_bootstrap.cc:654] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:58.728999 1188 consensus_queue.cc:260] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.729637 1188 ts_tablet_manager.cc:1398] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20240426 09:50:58.729962 1188 tablet_bootstrap.cc:492] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:50:58.730479 1186 tablet_bootstrap.cc:492] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:50:58.730564 1186 ts_tablet_manager.cc:1367] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:50:58.731077 1186 raft_consensus.cc:348] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.731199 1186 raft_consensus.cc:374] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:58.731230 1186 raft_consensus.cc:729] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:50:58.731154 1191 raft_consensus.cc:348] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:50:58.731323 1191 raft_consensus.cc:374] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:58.731367 1191 raft_consensus.cc:729] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:50:58.731343 1186 consensus_queue.cc:260] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.731608 1186 ts_tablet_manager.cc:1398] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:58.731794 1191 consensus_queue.cc:260] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:50:58.732098 1188 tablet_bootstrap.cc:654] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:58.732650 1191 ts_tablet_manager.cc:1398] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240426 09:50:58.732986 1191 tablet_bootstrap.cc:492] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:50:58.733175 1188 tablet_bootstrap.cc:492] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:50:58.733253 1188 ts_tablet_manager.cc:1367] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
W20240426 09:50:58.733327 512 tablet.cc:2369] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:50:58.733733 1188 raft_consensus.cc:348] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:50:58.733846 1188 raft_consensus.cc:374] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:58.733877 1188 raft_consensus.cc:729] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:50:58.734005 1188 consensus_queue.cc:260] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:50:58.734253 1188 ts_tablet_manager.cc:1398] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:58.734448 1191 tablet_bootstrap.cc:654] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:58.734928 1187 tablet_bootstrap.cc:492] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:50:58.735124 1187 ts_tablet_manager.cc:1367] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240426 09:50:58.735693 1191 tablet_bootstrap.cc:492] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:50:58.735785 1191 ts_tablet_manager.cc:1367] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240426 09:50:58.736301 1191 raft_consensus.cc:348] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.736302 1187 raft_consensus.cc:348] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:50:58.736407 1187 raft_consensus.cc:374] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:58.736407 1191 raft_consensus.cc:374] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:58.736439 1187 raft_consensus.cc:729] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:50:58.736439 1191 raft_consensus.cc:729] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:50:58.736563 1187 consensus_queue.cc:260] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:50:58.736570 1191 consensus_queue.cc:260] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.736830 1191 ts_tablet_manager.cc:1398] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:58.736843 1187 ts_tablet_manager.cc:1398] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:58.737003 1187 tablet_bootstrap.cc:492] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:50:58.737920 1187 tablet_bootstrap.cc:654] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:58.739073 1187 tablet_bootstrap.cc:492] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:50:58.739157 1187 ts_tablet_manager.cc:1367] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:58.739557 1187 raft_consensus.cc:348] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.739642 1187 raft_consensus.cc:374] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:58.739675 1187 raft_consensus.cc:729] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:50:58.739797 1187 consensus_queue.cc:260] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.740052 1187 ts_tablet_manager.cc:1398] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:58.783457 1197 raft_consensus.cc:482] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:58.783676 1197 raft_consensus.cc:504] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.784782 1197 leader_election.cc:288] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:50:58.788870 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2ecfdfe7ac44636b5d4b620c05ed1c9" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:50:58.788955 465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2ecfdfe7ac44636b5d4b620c05ed1c9" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:50:58.789197 328 raft_consensus.cc:2454] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:50:58.789218 465 raft_consensus.cc:2454] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:50:58.789657 532 leader_election.cc:302] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:58.789919 1197 raft_consensus.cc:2781] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:58.789988 1197 raft_consensus.cc:482] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:58.790019 1197 raft_consensus.cc:3037] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:58.790930 1197 raft_consensus.cc:504] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.791254 1197 leader_election.cc:288] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:50:58.791589 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2ecfdfe7ac44636b5d4b620c05ed1c9" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:50:58.791636 465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2ecfdfe7ac44636b5d4b620c05ed1c9" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:50:58.791690 328 raft_consensus.cc:3037] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:58.791764 465 raft_consensus.cc:3037] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:58.792651 465 raft_consensus.cc:2454] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:50:58.792961 328 raft_consensus.cc:2454] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:50:58.792966 532 leader_election.cc:302] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:58.793144 1197 raft_consensus.cc:2781] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:58.793362 1197 raft_consensus.cc:686] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:50:58.793686 1197 consensus_queue.cc:237] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.796458 32522 catalog_manager.cc:5484] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
W20240426 09:50:58.814980 375 tablet.cc:2369] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:50:58.827610 1199 raft_consensus.cc:482] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:58.827791 1199 raft_consensus.cc:504] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.828773 1199 leader_election.cc:288] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:50:58.832268 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "451f3108a6fb494da331dd6a6ec3e290" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:50:58.832356 465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "451f3108a6fb494da331dd6a6ec3e290" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:50:58.832407 328 raft_consensus.cc:2454] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:50:58.832490 465 raft_consensus.cc:2454] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:50:58.832767 32601 leader_election.cc:302] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:50:58.832985 1199 raft_consensus.cc:2781] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:58.833047 1199 raft_consensus.cc:482] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:58.833074 1199 raft_consensus.cc:3037] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:58.833892 1199 raft_consensus.cc:504] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.834214 1199 leader_election.cc:288] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:50:58.834543 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "451f3108a6fb494da331dd6a6ec3e290" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:50:58.834569 465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "451f3108a6fb494da331dd6a6ec3e290" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:50:58.834645 328 raft_consensus.cc:3037] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:58.834656 465 raft_consensus.cc:3037] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:58.835402 465 raft_consensus.cc:2454] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:50:58.835412 328 raft_consensus.cc:2454] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:50:58.835655 32601 leader_election.cc:302] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:50:58.835826 1199 raft_consensus.cc:2781] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:58.836005 1199 raft_consensus.cc:686] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:50:58.836323 1199 consensus_queue.cc:237] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:58.838654 32522 catalog_manager.cc:5484] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:50:58.886355 1199 raft_consensus.cc:482] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:58.886484 1199 raft_consensus.cc:504] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:50:58.886988 1199 leader_election.cc:288] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:50:58.887456 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:50:58.887619 328 raft_consensus.cc:2454] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:50:58.887916 32601 leader_election.cc:302] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:50:58.888105 1199 raft_consensus.cc:2781] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:58.888162 1199 raft_consensus.cc:482] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:58.888223 1199 raft_consensus.cc:3037] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:58.889158 1199 raft_consensus.cc:504] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:50:58.889519 1199 leader_election.cc:288] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:50:58.889885 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:50:58.890018 328 raft_consensus.cc:3037] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:58.890851 328 raft_consensus.cc:2454] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:50:58.890853 596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:50:58.890873 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:50:58.891044 595 raft_consensus.cc:3037] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:58.891777 32601 leader_election.cc:302] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:50:58.891939 1199 raft_consensus.cc:2781] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:58.892015 1199 raft_consensus.cc:686] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:50:58.892160 1199 consensus_queue.cc:237] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:50:58.892462 595 raft_consensus.cc:2454] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
W20240426 09:50:58.894106 32712 tablet.cc:2369] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 09:50:58.894042 32522 catalog_manager.cc:5484] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } }
W20240426 09:50:58.908109 643 tablet.cc:2369] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240426 09:50:59.040860 1212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:59.041118 1212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:59.044243 1212 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240426 09:50:59.144024 465 raft_consensus.cc:1261] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.144078 328 raft_consensus.cc:1261] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.144907 1205 consensus_queue.cc:1035] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.145413 1197 consensus_queue.cc:1035] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.149168 464 raft_consensus.cc:1261] T 451f3108a6fb494da331dd6a6ec3e290 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.149793 1209 consensus_queue.cc:1035] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.150256 326 raft_consensus.cc:1261] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.150440 327 raft_consensus.cc:1261] T 451f3108a6fb494da331dd6a6ec3e290 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.150482 595 raft_consensus.cc:1261] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.152456 1209 consensus_queue.cc:1035] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.152681 1199 consensus_queue.cc:1035] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.152843 1209 consensus_queue.cc:1035] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.156596 1241 mvcc.cc:205] Tried to move back new op lower bound from 7021056242246795264 to 7021056240821469184. Current Snapshot: MvccSnapshot[applied={T|T < 7021056242246795264}]
I20240426 09:50:59.165612 1252 mvcc.cc:205] Tried to move back new op lower bound from 7021056242246795264 to 7021056240821469184. Current Snapshot: MvccSnapshot[applied={T|T < 7021056242246795264}]
I20240426 09:50:59.171516 1243 mvcc.cc:205] Tried to move back new op lower bound from 7021056242268925952 to 7021056240995971072. Current Snapshot: MvccSnapshot[applied={T|T < 7021056242268925952}]
I20240426 09:50:59.181443 1245 mvcc.cc:205] Tried to move back new op lower bound from 7021056242246795264 to 7021056240821469184. Current Snapshot: MvccSnapshot[applied={T|T < 7021056242246795264}]
INSERT report
rows total: 2000
time total: 130.691 ms
time per row: 0.0653455 ms
I20240426 09:50:59.222525 32517 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:50:59.227892 32645 tablet_service.cc:1467] Processing CreateTablet for tablet 99333b8e2108485cb56bee5d5fd22d25 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a586b43aa5d64235894a2b154fb34b95]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.227979 445 tablet_service.cc:1467] Processing CreateTablet for tablet 99333b8e2108485cb56bee5d5fd22d25 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a586b43aa5d64235894a2b154fb34b95]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.228082 576 tablet_service.cc:1467] Processing CreateTablet for tablet 99333b8e2108485cb56bee5d5fd22d25 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a586b43aa5d64235894a2b154fb34b95]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.228214 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99333b8e2108485cb56bee5d5fd22d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.228268 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99333b8e2108485cb56bee5d5fd22d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.228319 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99333b8e2108485cb56bee5d5fd22d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.229880 1186 tablet_bootstrap.cc:492] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:50:59.230417 1188 tablet_bootstrap.cc:492] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:50:59.230724 1186 tablet_bootstrap.cc:654] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.230737 301 tablet_service.cc:1467] Processing CreateTablet for tablet 3abebac937b44862950b2f1957cca770 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a586b43aa5d64235894a2b154fb34b95]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.231063 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3abebac937b44862950b2f1957cca770. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.231307 1188 tablet_bootstrap.cc:654] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.231575 445 tablet_service.cc:1467] Processing CreateTablet for tablet 713fa4d62643470285727c0d0092676f (DEFAULT_TABLE table=rebalance_test_table_1 [id=a586b43aa5d64235894a2b154fb34b95]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.231801 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 713fa4d62643470285727c0d0092676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.231824 1186 tablet_bootstrap.cc:492] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:50:59.231896 1186 ts_tablet_manager.cc:1367] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:50:59.231935 1191 tablet_bootstrap.cc:492] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:50:59.232378 1186 raft_consensus.cc:348] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.232472 1186 raft_consensus.cc:374] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.232502 1186 raft_consensus.cc:729] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.232625 1186 consensus_queue.cc:260] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.232753 1191 tablet_bootstrap.cc:654] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.232893 1186 ts_tablet_manager.cc:1398] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:59.233393 576 tablet_service.cc:1467] Processing CreateTablet for tablet 713fa4d62643470285727c0d0092676f (DEFAULT_TABLE table=rebalance_test_table_1 [id=a586b43aa5d64235894a2b154fb34b95]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.233407 444 tablet_service.cc:1467] Processing CreateTablet for tablet 3abebac937b44862950b2f1957cca770 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a586b43aa5d64235894a2b154fb34b95]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.233585 1187 tablet_bootstrap.cc:492] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:50:59.233599 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3abebac937b44862950b2f1957cca770. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.233599 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 713fa4d62643470285727c0d0092676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.234263 1187 tablet_bootstrap.cc:654] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.235309 1187 tablet_bootstrap.cc:492] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:50:59.235384 1187 ts_tablet_manager.cc:1367] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:50:59.235759 1186 tablet_bootstrap.cc:492] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:50:59.235807 1187 raft_consensus.cc:348] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.235909 1187 raft_consensus.cc:374] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.235944 1187 raft_consensus.cc:729] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.236083 1187 consensus_queue.cc:260] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.236346 1187 ts_tablet_manager.cc:1398] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:59.236472 32644 tablet_service.cc:1467] Processing CreateTablet for tablet 3abebac937b44862950b2f1957cca770 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a586b43aa5d64235894a2b154fb34b95]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.236627 1186 tablet_bootstrap.cc:654] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.236747 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3abebac937b44862950b2f1957cca770. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.238039 1188 tablet_bootstrap.cc:492] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:50:59.238101 1191 tablet_bootstrap.cc:492] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:50:59.238126 1188 ts_tablet_manager.cc:1367] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:50:59.238183 1191 ts_tablet_manager.cc:1367] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240426 09:50:59.238562 1191 raft_consensus.cc:348] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.238570 1188 raft_consensus.cc:348] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.238646 1191 raft_consensus.cc:374] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.238675 1188 raft_consensus.cc:374] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.238680 1191 raft_consensus.cc:729] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.238723 1188 raft_consensus.cc:729] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.238842 1191 consensus_queue.cc:260] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.238893 1188 consensus_queue.cc:260] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.239140 1191 ts_tablet_manager.cc:1398] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:59.239183 1188 ts_tablet_manager.cc:1398] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:59.239279 1191 tablet_bootstrap.cc:492] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:50:59.239358 1188 tablet_bootstrap.cc:492] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:50:59.240150 1191 tablet_bootstrap.cc:654] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.240634 1188 tablet_bootstrap.cc:654] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.240782 32645 tablet_service.cc:1467] Processing CreateTablet for tablet 713fa4d62643470285727c0d0092676f (DEFAULT_TABLE table=rebalance_test_table_1 [id=a586b43aa5d64235894a2b154fb34b95]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.240998 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 713fa4d62643470285727c0d0092676f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.241632 1191 tablet_bootstrap.cc:492] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:50:59.241698 1191 ts_tablet_manager.cc:1367] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:50:59.242007 1191 raft_consensus.cc:348] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.242074 1191 raft_consensus.cc:374] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.242094 1191 raft_consensus.cc:729] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.242365 1191 consensus_queue.cc:260] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.243194 1191 ts_tablet_manager.cc:1398] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:50:59.243362 1191 tablet_bootstrap.cc:492] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:50:59.243682 1186 tablet_bootstrap.cc:492] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:50:59.243782 1186 ts_tablet_manager.cc:1367] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240426 09:50:59.244346 1186 raft_consensus.cc:348] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.244464 1186 raft_consensus.cc:374] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.244498 1186 raft_consensus.cc:729] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.244634 1186 consensus_queue.cc:260] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.245029 1186 ts_tablet_manager.cc:1398] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:59.245116 1191 tablet_bootstrap.cc:654] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.245398 1186 tablet_bootstrap.cc:492] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:50:59.246366 1186 tablet_bootstrap.cc:654] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.247679 1188 tablet_bootstrap.cc:492] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:50:59.247759 1186 tablet_bootstrap.cc:492] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:50:59.247783 1188 ts_tablet_manager.cc:1367] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:50:59.247830 1186 ts_tablet_manager.cc:1367] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:50:59.248139 1186 raft_consensus.cc:348] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.248219 1186 raft_consensus.cc:374] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.248162 1188 raft_consensus.cc:348] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.248239 1186 raft_consensus.cc:729] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.248252 1188 raft_consensus.cc:374] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.248296 1188 raft_consensus.cc:729] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.248332 1186 consensus_queue.cc:260] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.248423 1188 consensus_queue.cc:260] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.248580 1186 ts_tablet_manager.cc:1398] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:59.248762 1188 ts_tablet_manager.cc:1398] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:59.249493 1191 tablet_bootstrap.cc:492] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:50:59.249588 1191 ts_tablet_manager.cc:1367] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240426 09:50:59.250037 1191 raft_consensus.cc:348] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.250150 1191 raft_consensus.cc:374] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.250205 1191 raft_consensus.cc:729] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.250341 1191 consensus_queue.cc:260] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.250613 1191 ts_tablet_manager.cc:1398] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240426 09:50:59.250833 1194 raft_consensus.cc:482] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:59.250952 1194 raft_consensus.cc:504] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.252025 1194 leader_election.cc:288] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:50:59.256264 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abebac937b44862950b2f1957cca770" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:50:59.256291 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abebac937b44862950b2f1957cca770" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:50:59.256417 464 raft_consensus.cc:2454] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:50:59.256493 32665 raft_consensus.cc:2454] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:50:59.256783 32732 leader_election.cc:302] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:50:59.256839 1261 raft_consensus.cc:482] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:59.256904 1261 raft_consensus.cc:504] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.257037 1194 raft_consensus.cc:2781] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:59.257099 1194 raft_consensus.cc:482] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:59.257138 1194 raft_consensus.cc:3037] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.257301 1261 leader_election.cc:288] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:50:59.257665 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:50:59.257805 595 raft_consensus.cc:2454] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:50:59.258082 32598 leader_election.cc:302] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:59.258224 1261 raft_consensus.cc:2781] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:59.258277 1261 raft_consensus.cc:482] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:59.258226 1194 raft_consensus.cc:504] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.258332 1261 raft_consensus.cc:3037] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.258595 1194 leader_election.cc:288] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:50:59.259027 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abebac937b44862950b2f1957cca770" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:50:59.259070 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:50:59.259155 32665 raft_consensus.cc:3037] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.259171 464 raft_consensus.cc:2454] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:50:59.259330 1261 raft_consensus.cc:504] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.259632 1261 leader_election.cc:288] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:50:59.259893 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:50:59.259985 464 raft_consensus.cc:3037] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.260046 32665 raft_consensus.cc:2454] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:50:59.260293 32732 leader_election.cc:302] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:50:59.260418 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:50:59.260452 1194 raft_consensus.cc:2781] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:59.260524 595 raft_consensus.cc:3037] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.260501 465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abebac937b44862950b2f1957cca770" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:50:59.260601 465 raft_consensus.cc:3037] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.260664 1194 raft_consensus.cc:686] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Leader. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:50:59.260986 464 raft_consensus.cc:2454] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:50:59.260968 1194 consensus_queue.cc:237] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.261421 465 raft_consensus.cc:2454] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:50:59.261423 595 raft_consensus.cc:2454] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:50:59.261715 32598 leader_election.cc:302] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:59.261969 1261 raft_consensus.cc:2781] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:59.262056 1261 raft_consensus.cc:686] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:50:59.262189 1261 consensus_queue.cc:237] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.263247 32517 catalog_manager.cc:5484] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130). New cstate: current_term: 1 leader_uuid: "07593a4e1c0242a1b52bb96338b63c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:50:59.263762 32522 catalog_manager.cc:5484] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:50:59.264429 1261 raft_consensus.cc:482] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:59.264508 1261 raft_consensus.cc:504] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.264850 1261 leader_election.cc:288] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:50:59.265239 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713fa4d62643470285727c0d0092676f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:50:59.265359 595 raft_consensus.cc:2454] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:50:59.265489 465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713fa4d62643470285727c0d0092676f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:50:59.265610 465 raft_consensus.cc:2454] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:50:59.265632 32598 leader_election.cc:302] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:59.266002 1261 raft_consensus.cc:2781] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:59.266059 1261 raft_consensus.cc:482] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:59.266108 1261 raft_consensus.cc:3037] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.266954 1261 raft_consensus.cc:504] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.267230 1261 leader_election.cc:288] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:50:59.267529 465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713fa4d62643470285727c0d0092676f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:50:59.267524 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713fa4d62643470285727c0d0092676f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:50:59.267632 465 raft_consensus.cc:3037] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.267629 595 raft_consensus.cc:3037] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.268309 595 raft_consensus.cc:2454] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:50:59.268383 465 raft_consensus.cc:2454] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:50:59.268596 32598 leader_election.cc:302] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:59.268733 1261 raft_consensus.cc:2781] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:59.268821 1261 raft_consensus.cc:686] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:50:59.268936 1261 consensus_queue.cc:237] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:50:59.270334 32522 catalog_manager.cc:5484] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
W20240426 09:50:59.381505 1286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:59.381785 1286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:59.384881 1286 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240426 09:50:59.467365 465 raft_consensus.cc:1261] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.468063 596 raft_consensus.cc:1261] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.468205 465 raft_consensus.cc:1261] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.468300 1247 consensus_queue.cc:1035] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.468554 1261 consensus_queue.cc:1035] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.469676 595 raft_consensus.cc:1261] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.473243 1199 consensus_queue.cc:1035] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.473325 1261 consensus_queue.cc:1035] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.473920 32665 raft_consensus.cc:1261] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:59.474701 465 raft_consensus.cc:1261] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240426 09:50:59.474977 1315 consensus_queue.cc:1035] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.497119 1314 consensus_queue.cc:1035] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 104.21 ms
time per row: 0.0521052 ms
I20240426 09:50:59.530428 32520 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:50:59.536036 301 tablet_service.cc:1467] Processing CreateTablet for tablet 679d7a8559df4e6fbae5e44dc3b2e227 (DEFAULT_TABLE table=rebalance_test_table_2 [id=def26891a6114f36b72b7e5e6aa46da3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.536377 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 679d7a8559df4e6fbae5e44dc3b2e227. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.536854 559 tablet_service.cc:1467] Processing CreateTablet for tablet d12dc2eddc8c402fb0f12c97410d9af2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=def26891a6114f36b72b7e5e6aa46da3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.537106 559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d12dc2eddc8c402fb0f12c97410d9af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.537129 32644 tablet_service.cc:1467] Processing CreateTablet for tablet 9d752855971a4015b1ebc62993ea9b22 (DEFAULT_TABLE table=rebalance_test_table_2 [id=def26891a6114f36b72b7e5e6aa46da3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.537392 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d752855971a4015b1ebc62993ea9b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.537360 32645 tablet_service.cc:1467] Processing CreateTablet for tablet 679d7a8559df4e6fbae5e44dc3b2e227 (DEFAULT_TABLE table=rebalance_test_table_2 [id=def26891a6114f36b72b7e5e6aa46da3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.537582 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 679d7a8559df4e6fbae5e44dc3b2e227. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.538604 1187 tablet_bootstrap.cc:492] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:50:59.539279 1188 tablet_bootstrap.cc:492] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:50:59.539577 1187 tablet_bootstrap.cc:654] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.539604 1191 tablet_bootstrap.cc:492] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:50:59.539863 445 tablet_service.cc:1467] Processing CreateTablet for tablet d12dc2eddc8c402fb0f12c97410d9af2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=def26891a6114f36b72b7e5e6aa46da3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.540132 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d12dc2eddc8c402fb0f12c97410d9af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.540148 1188 tablet_bootstrap.cc:654] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.540529 1191 tablet_bootstrap.cc:654] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.541106 1187 tablet_bootstrap.cc:492] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:50:59.541188 1187 ts_tablet_manager.cc:1367] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:50:59.541648 1187 raft_consensus.cc:348] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.541752 1187 raft_consensus.cc:374] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.541782 1187 raft_consensus.cc:729] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.541785 1188 tablet_bootstrap.cc:492] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:50:59.541873 1188 ts_tablet_manager.cc:1367] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:50:59.541837 576 tablet_service.cc:1467] Processing CreateTablet for tablet 679d7a8559df4e6fbae5e44dc3b2e227 (DEFAULT_TABLE table=rebalance_test_table_2 [id=def26891a6114f36b72b7e5e6aa46da3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.541918 1187 consensus_queue.cc:260] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.542052 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 679d7a8559df4e6fbae5e44dc3b2e227. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.542215 1187 ts_tablet_manager.cc:1398] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:59.542265 1186 tablet_bootstrap.cc:492] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:50:59.542302 1188 raft_consensus.cc:348] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.542393 1188 raft_consensus.cc:374] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.542421 1188 raft_consensus.cc:729] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.542531 1188 consensus_queue.cc:260] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.543197 1186 tablet_bootstrap.cc:654] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.543715 1191 tablet_bootstrap.cc:492] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:50:59.543797 1191 ts_tablet_manager.cc:1367] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240426 09:50:59.544253 1191 raft_consensus.cc:348] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.544353 1191 raft_consensus.cc:374] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.544409 1191 raft_consensus.cc:729] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.544540 1191 consensus_queue.cc:260] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.544791 1191 ts_tablet_manager.cc:1398] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240426 09:50:59.544792 301 tablet_service.cc:1467] Processing CreateTablet for tablet d12dc2eddc8c402fb0f12c97410d9af2 (DEFAULT_TABLE table=rebalance_test_table_2 [id=def26891a6114f36b72b7e5e6aa46da3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.544970 1191 tablet_bootstrap.cc:492] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:50:59.545022 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d12dc2eddc8c402fb0f12c97410d9af2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.545991 1186 tablet_bootstrap.cc:492] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:50:59.545951 308 tablet_service.cc:1467] Processing CreateTablet for tablet 9d752855971a4015b1ebc62993ea9b22 (DEFAULT_TABLE table=rebalance_test_table_2 [id=def26891a6114f36b72b7e5e6aa46da3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.546061 1186 ts_tablet_manager.cc:1367] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:50:59.546175 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d752855971a4015b1ebc62993ea9b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.546499 1186 raft_consensus.cc:348] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.546605 1186 raft_consensus.cc:374] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.546634 1186 raft_consensus.cc:729] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.546782 1186 consensus_queue.cc:260] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.546893 1188 ts_tablet_manager.cc:1398] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:50:59.547040 1186 ts_tablet_manager.cc:1398] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:59.547071 1187 tablet_bootstrap.cc:492] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:50:59.547070 1188 tablet_bootstrap.cc:492] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:50:59.547935 1187 tablet_bootstrap.cc:654] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.548486 1188 tablet_bootstrap.cc:654] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.537050 574 tablet_service.cc:1467] Processing CreateTablet for tablet 9d752855971a4015b1ebc62993ea9b22 (DEFAULT_TABLE table=rebalance_test_table_2 [id=def26891a6114f36b72b7e5e6aa46da3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:50:59.549607 1188 tablet_bootstrap.cc:492] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:50:59.549634 1191 tablet_bootstrap.cc:654] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.549683 1188 ts_tablet_manager.cc:1367] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:50:59.549760 574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d752855971a4015b1ebc62993ea9b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:50:59.550150 1188 raft_consensus.cc:348] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.550267 1188 raft_consensus.cc:374] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.550305 1188 raft_consensus.cc:729] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.550452 1188 consensus_queue.cc:260] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.550940 1188 ts_tablet_manager.cc:1398] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240426 09:50:59.551144 1187 tablet_bootstrap.cc:492] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:50:59.551203 1187 ts_tablet_manager.cc:1367] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240426 09:50:59.551271 1191 tablet_bootstrap.cc:492] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:50:59.551362 1191 ts_tablet_manager.cc:1367] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240426 09:50:59.551504 1187 raft_consensus.cc:348] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.551584 1187 raft_consensus.cc:374] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.551604 1187 raft_consensus.cc:729] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.551697 1187 consensus_queue.cc:260] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.551831 1191 raft_consensus.cc:348] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.551932 1191 raft_consensus.cc:374] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.551959 1187 ts_tablet_manager.cc:1398] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:59.551973 1191 raft_consensus.cc:729] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.552109 1187 tablet_bootstrap.cc:492] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:50:59.552101 1191 consensus_queue.cc:260] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.552381 1191 ts_tablet_manager.cc:1398] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240426 09:50:59.552985 1187 tablet_bootstrap.cc:654] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.553860 1188 tablet_bootstrap.cc:492] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:50:59.554108 1187 tablet_bootstrap.cc:492] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:50:59.554185 1187 ts_tablet_manager.cc:1367] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:50:59.554587 1187 raft_consensus.cc:348] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.554688 1187 raft_consensus.cc:374] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.554725 1187 raft_consensus.cc:729] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.554805 1188 tablet_bootstrap.cc:654] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:50:59.554865 1187 consensus_queue.cc:260] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.555148 1187 ts_tablet_manager.cc:1398] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:50:59.556031 1188 tablet_bootstrap.cc:492] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:50:59.556126 1188 ts_tablet_manager.cc:1367] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240426 09:50:59.556484 1188 raft_consensus.cc:348] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.556605 1188 raft_consensus.cc:374] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:50:59.556640 1188 raft_consensus.cc:729] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:50:59.556771 1188 consensus_queue.cc:260] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.557054 1188 ts_tablet_manager.cc:1398] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:50:59.565627 1197 raft_consensus.cc:482] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:59.565749 1197 raft_consensus.cc:504] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.566149 1197 leader_election.cc:288] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:50:59.566569 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d12dc2eddc8c402fb0f12c97410d9af2" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:50:59.566593 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d12dc2eddc8c402fb0f12c97410d9af2" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:50:59.566701 464 raft_consensus.cc:2454] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:50:59.566833 328 raft_consensus.cc:2454] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:50:59.567175 532 leader_election.cc:302] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:59.567363 1197 raft_consensus.cc:2781] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:59.567428 1197 raft_consensus.cc:482] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:59.567467 1197 raft_consensus.cc:3037] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.568405 1197 raft_consensus.cc:504] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.568684 1197 leader_election.cc:288] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:50:59.569021 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d12dc2eddc8c402fb0f12c97410d9af2" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:50:59.569034 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d12dc2eddc8c402fb0f12c97410d9af2" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:50:59.569144 464 raft_consensus.cc:3037] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.569151 328 raft_consensus.cc:3037] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.569851 464 raft_consensus.cc:2454] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:50:59.569916 328 raft_consensus.cc:2454] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:50:59.570118 532 leader_election.cc:302] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:59.570251 1197 raft_consensus.cc:2781] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:59.570340 1197 raft_consensus.cc:686] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:50:59.570492 1197 consensus_queue.cc:237] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:50:59.572041 32518 catalog_manager.cc:5484] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:50:59.682849 1197 raft_consensus.cc:482] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:59.683023 1197 raft_consensus.cc:504] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.683833 1197 leader_election.cc:288] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:50:59.684846 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d752855971a4015b1ebc62993ea9b22" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:50:59.685000 328 raft_consensus.cc:2454] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:50:59.685390 532 leader_election.cc:302] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:59.686859 1197 raft_consensus.cc:2781] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:59.686928 1197 raft_consensus.cc:482] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:59.686977 1197 raft_consensus.cc:3037] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.687896 1197 raft_consensus.cc:504] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.688161 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d752855971a4015b1ebc62993ea9b22" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:50:59.688284 32665 raft_consensus.cc:2454] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:50:59.688699 1197 leader_election.cc:288] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:50:59.689045 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d752855971a4015b1ebc62993ea9b22" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:50:59.689178 328 raft_consensus.cc:3037] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.689242 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d752855971a4015b1ebc62993ea9b22" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:50:59.689343 32665 raft_consensus.cc:3037] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.689970 328 raft_consensus.cc:2454] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:50:59.690053 32665 raft_consensus.cc:2454] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:50:59.690280 532 leader_election.cc:302] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:50:59.690435 1197 raft_consensus.cc:2781] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:59.690552 1197 raft_consensus.cc:686] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:50:59.690703 1197 consensus_queue.cc:237] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.692435 32518 catalog_manager.cc:5484] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
I20240426 09:50:59.757652 1247 raft_consensus.cc:482] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:50:59.757833 1247 raft_consensus.cc:504] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.758267 1247 leader_election.cc:288] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:50:59.758726 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679d7a8559df4e6fbae5e44dc3b2e227" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:50:59.758759 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679d7a8559df4e6fbae5e44dc3b2e227" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:50:59.758889 328 raft_consensus.cc:2454] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:50:59.758949 595 raft_consensus.cc:2454] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:50:59.759231 32601 leader_election.cc:302] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:50:59.759405 1247 raft_consensus.cc:2781] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:50:59.759460 1247 raft_consensus.cc:482] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:50:59.759505 1247 raft_consensus.cc:3037] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.760342 1247 raft_consensus.cc:504] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.760685 1247 leader_election.cc:288] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:50:59.761021 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679d7a8559df4e6fbae5e44dc3b2e227" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:50:59.761024 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679d7a8559df4e6fbae5e44dc3b2e227" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:50:59.761119 328 raft_consensus.cc:3037] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.761130 595 raft_consensus.cc:3037] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:50:59.761876 595 raft_consensus.cc:2454] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:50:59.761885 328 raft_consensus.cc:2454] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:50:59.762151 32601 leader_election.cc:302] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:50:59.762300 1247 raft_consensus.cc:2781] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:50:59.762392 1247 raft_consensus.cc:686] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:50:59.762534 1247 consensus_queue.cc:237] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:50:59.764261 32518 catalog_manager.cc:5484] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
W20240426 09:50:59.885218 1336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:50:59.885485 1336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:50:59.888293 1336 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240426 09:50:59.967278 595 raft_consensus.cc:1261] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.967394 328 raft_consensus.cc:1261] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.967828 1247 consensus_queue.cc:1035] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.968464 328 raft_consensus.cc:1261] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.968498 32665 raft_consensus.cc:1261] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.968755 1247 consensus_queue.cc:1035] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.969257 1197 consensus_queue.cc:1035] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.973373 1205 consensus_queue.cc:1035] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.974442 326 raft_consensus.cc:1261] T d12dc2eddc8c402fb0f12c97410d9af2 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.974880 464 raft_consensus.cc:1261] T d12dc2eddc8c402fb0f12c97410d9af2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:50:59.974884 1365 consensus_queue.cc:1035] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:50:59.977653 1197 consensus_queue.cc:1035] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 100.387 ms
time per row: 0.0501937 ms
I20240426 09:51:00.034011 32518 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:51:00.039458 32645 tablet_service.cc:1467] Processing CreateTablet for tablet 76ada11863ac4cce802de521b842be2d (DEFAULT_TABLE table=rebalance_test_table_3 [id=0c8725737eec44d9a48aae0bb2b33a98]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.039433 308 tablet_service.cc:1467] Processing CreateTablet for tablet 76ada11863ac4cce802de521b842be2d (DEFAULT_TABLE table=rebalance_test_table_3 [id=0c8725737eec44d9a48aae0bb2b33a98]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.039486 445 tablet_service.cc:1467] Processing CreateTablet for tablet 76ada11863ac4cce802de521b842be2d (DEFAULT_TABLE table=rebalance_test_table_3 [id=0c8725737eec44d9a48aae0bb2b33a98]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.039745 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76ada11863ac4cce802de521b842be2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.039750 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76ada11863ac4cce802de521b842be2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.039780 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76ada11863ac4cce802de521b842be2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.041505 1186 tablet_bootstrap.cc:492] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:00.041833 1191 tablet_bootstrap.cc:492] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:00.041833 1187 tablet_bootstrap.cc:492] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:00.042389 1186 tablet_bootstrap.cc:654] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.042443 574 tablet_service.cc:1467] Processing CreateTablet for tablet 4584b80c6d7e4feca28d7b321bca613f (DEFAULT_TABLE table=rebalance_test_table_3 [id=0c8725737eec44d9a48aae0bb2b33a98]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.042673 1187 tablet_bootstrap.cc:654] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.042706 574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4584b80c6d7e4feca28d7b321bca613f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.043066 1191 tablet_bootstrap.cc:654] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.043814 444 tablet_service.cc:1467] Processing CreateTablet for tablet 3c8353b8e1494e33a069eb6921b81ba5 (DEFAULT_TABLE table=rebalance_test_table_3 [id=0c8725737eec44d9a48aae0bb2b33a98]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.044035 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c8353b8e1494e33a069eb6921b81ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.044134 1191 tablet_bootstrap.cc:492] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:00.044227 1191 ts_tablet_manager.cc:1367] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:00.044518 1188 tablet_bootstrap.cc:492] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:00.044679 576 tablet_service.cc:1467] Processing CreateTablet for tablet 3c8353b8e1494e33a069eb6921b81ba5 (DEFAULT_TABLE table=rebalance_test_table_3 [id=0c8725737eec44d9a48aae0bb2b33a98]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.044679 1191 raft_consensus.cc:348] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.044783 1191 raft_consensus.cc:374] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.044813 1191 raft_consensus.cc:729] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.044909 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c8353b8e1494e33a069eb6921b81ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.044939 1191 consensus_queue.cc:260] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.045198 1191 ts_tablet_manager.cc:1398] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:00.045310 1188 tablet_bootstrap.cc:654] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.046350 1188 tablet_bootstrap.cc:492] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:00.046442 1188 ts_tablet_manager.cc:1367] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240426 09:51:00.046873 32645 tablet_service.cc:1467] Processing CreateTablet for tablet 3c8353b8e1494e33a069eb6921b81ba5 (DEFAULT_TABLE table=rebalance_test_table_3 [id=0c8725737eec44d9a48aae0bb2b33a98]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.046913 1188 raft_consensus.cc:348] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.047014 1188 raft_consensus.cc:374] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.047052 1188 raft_consensus.cc:729] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.047106 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c8353b8e1494e33a069eb6921b81ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.047165 1188 consensus_queue.cc:260] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.047420 1188 ts_tablet_manager.cc:1398] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.047611 1188 tablet_bootstrap.cc:492] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:00.047833 1186 tablet_bootstrap.cc:492] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:00.047910 1186 ts_tablet_manager.cc:1367] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:51:00.048385 1186 raft_consensus.cc:348] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.048449 1188 tablet_bootstrap.cc:654] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.048475 1186 raft_consensus.cc:374] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.048506 1186 raft_consensus.cc:729] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.048619 1186 consensus_queue.cc:260] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.048710 445 tablet_service.cc:1467] Processing CreateTablet for tablet 4584b80c6d7e4feca28d7b321bca613f (DEFAULT_TABLE table=rebalance_test_table_3 [id=0c8725737eec44d9a48aae0bb2b33a98]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.048900 1186 ts_tablet_manager.cc:1398] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.048962 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4584b80c6d7e4feca28d7b321bca613f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.049098 1186 tablet_bootstrap.cc:492] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:00.049453 1191 tablet_bootstrap.cc:492] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:00.049979 1186 tablet_bootstrap.cc:654] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.050359 1191 tablet_bootstrap.cc:654] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.051118 308 tablet_service.cc:1467] Processing CreateTablet for tablet 4584b80c6d7e4feca28d7b321bca613f (DEFAULT_TABLE table=rebalance_test_table_3 [id=0c8725737eec44d9a48aae0bb2b33a98]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.051388 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4584b80c6d7e4feca28d7b321bca613f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.051717 1186 tablet_bootstrap.cc:492] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:00.051795 1186 ts_tablet_manager.cc:1367] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:51:00.052213 1186 raft_consensus.cc:348] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.052304 1186 raft_consensus.cc:374] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.052335 1186 raft_consensus.cc:729] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.052450 1186 consensus_queue.cc:260] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.052690 1186 ts_tablet_manager.cc:1398] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.054260 1186 tablet_bootstrap.cc:492] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:00.054997 1191 tablet_bootstrap.cc:492] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:00.055089 1191 ts_tablet_manager.cc:1367] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:51:00.055527 1191 raft_consensus.cc:348] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.055635 1186 tablet_bootstrap.cc:654] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.055636 1191 raft_consensus.cc:374] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.055672 1191 raft_consensus.cc:729] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.055792 1191 consensus_queue.cc:260] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.056035 1191 ts_tablet_manager.cc:1398] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:00.057065 1186 tablet_bootstrap.cc:492] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:00.057157 1186 ts_tablet_manager.cc:1367] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:51:00.057590 1186 raft_consensus.cc:348] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.057691 1186 raft_consensus.cc:374] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.057729 1186 raft_consensus.cc:729] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.057842 1186 consensus_queue.cc:260] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.058092 1186 ts_tablet_manager.cc:1398] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:51:00.058647 1187 tablet_bootstrap.cc:492] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:00.058741 1187 ts_tablet_manager.cc:1367] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20240426 09:51:00.059350 1188 tablet_bootstrap.cc:492] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:00.059293 1187 raft_consensus.cc:348] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.059398 1187 raft_consensus.cc:374] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.059440 1188 ts_tablet_manager.cc:1367] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240426 09:51:00.059435 1187 raft_consensus.cc:729] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.059576 1187 consensus_queue.cc:260] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.059844 1187 ts_tablet_manager.cc:1398] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.060017 1187 tablet_bootstrap.cc:492] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:00.060027 1188 raft_consensus.cc:348] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.060135 1188 raft_consensus.cc:374] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.060305 1188 raft_consensus.cc:729] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.060477 1188 consensus_queue.cc:260] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.060775 1188 ts_tablet_manager.cc:1398] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.061005 1187 tablet_bootstrap.cc:654] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.062320 1187 tablet_bootstrap.cc:492] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:00.062404 1187 ts_tablet_manager.cc:1367] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:51:00.062872 1187 raft_consensus.cc:348] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.062974 1187 raft_consensus.cc:374] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.063006 1187 raft_consensus.cc:729] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.063156 1187 consensus_queue.cc:260] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.063525 1187 ts_tablet_manager.cc:1398] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.102684 1315 raft_consensus.cc:482] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:00.102854 1315 raft_consensus.cc:504] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.103413 1315 leader_election.cc:288] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:00.103787 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4584b80c6d7e4feca28d7b321bca613f" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:00.103996 464 raft_consensus.cc:2454] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:00.104334 32732 leader_election.cc:302] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:00.104780 1315 raft_consensus.cc:2781] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:00.104873 1315 raft_consensus.cc:482] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:00.104911 1315 raft_consensus.cc:3037] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.105780 1315 raft_consensus.cc:504] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.106115 1315 leader_election.cc:288] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:00.106443 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4584b80c6d7e4feca28d7b321bca613f" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:00.106554 464 raft_consensus.cc:3037] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.107404 464 raft_consensus.cc:2454] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:00.107671 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4584b80c6d7e4feca28d7b321bca613f" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:00.107749 596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4584b80c6d7e4feca28d7b321bca613f" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:00.107811 595 raft_consensus.cc:2454] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:00.107882 596 raft_consensus.cc:3037] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.108062 32732 leader_election.cc:302] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:00.108230 1315 raft_consensus.cc:2781] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:00.108306 1315 raft_consensus.cc:686] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Leader. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:00.108456 1315 consensus_queue.cc:237] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.108739 596 raft_consensus.cc:2454] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:00.110261 32521 catalog_manager.cc:5484] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130). New cstate: current_term: 1 leader_uuid: "07593a4e1c0242a1b52bb96338b63c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } }
I20240426 09:51:00.216897 1365 raft_consensus.cc:482] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:00.217032 1365 raft_consensus.cc:504] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.217440 1365 leader_election.cc:288] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:00.217890 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c8353b8e1494e33a069eb6921b81ba5" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:00.217957 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c8353b8e1494e33a069eb6921b81ba5" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:00.218019 464 raft_consensus.cc:2454] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:00.218116 32665 raft_consensus.cc:2454] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:00.218308 532 leader_election.cc:302] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:00.218487 1365 raft_consensus.cc:2781] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:00.218554 1365 raft_consensus.cc:482] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:00.218605 1365 raft_consensus.cc:3037] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.219457 1365 raft_consensus.cc:504] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.219758 1365 leader_election.cc:288] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:00.220113 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c8353b8e1494e33a069eb6921b81ba5" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:00.220150 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c8353b8e1494e33a069eb6921b81ba5" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:00.220223 464 raft_consensus.cc:3037] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.220244 32665 raft_consensus.cc:3037] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.220927 464 raft_consensus.cc:2454] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:00.220992 32665 raft_consensus.cc:2454] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:00.221271 532 leader_election.cc:302] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:00.221446 1365 raft_consensus.cc:2781] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:00.221529 1365 raft_consensus.cc:686] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:00.221674 1365 consensus_queue.cc:237] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.223263 32521 catalog_manager.cc:5484] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
I20240426 09:51:00.295656 1247 raft_consensus.cc:482] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:00.295815 1247 raft_consensus.cc:504] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.296291 1247 leader_election.cc:288] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:00.296769 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76ada11863ac4cce802de521b842be2d" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:00.296824 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76ada11863ac4cce802de521b842be2d" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:00.296936 464 raft_consensus.cc:2454] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:00.296948 326 raft_consensus.cc:2454] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:00.297247 32601 leader_election.cc:302] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:51:00.297405 1247 raft_consensus.cc:2781] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:00.297459 1247 raft_consensus.cc:482] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:00.297504 1247 raft_consensus.cc:3037] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.298310 1247 raft_consensus.cc:504] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.298609 1247 leader_election.cc:288] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:00.298990 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76ada11863ac4cce802de521b842be2d" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:00.298990 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76ada11863ac4cce802de521b842be2d" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:00.299110 326 raft_consensus.cc:3037] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.299111 464 raft_consensus.cc:3037] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.300108 326 raft_consensus.cc:2454] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:00.300108 464 raft_consensus.cc:2454] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:00.300418 32601 leader_election.cc:302] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:51:00.300580 1247 raft_consensus.cc:2781] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:00.300665 1247 raft_consensus.cc:686] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:00.300805 1247 consensus_queue.cc:237] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.302495 32521 catalog_manager.cc:5484] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
W20240426 09:51:00.452812 1390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:00.453070 1390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:00.456090 1390 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240426 09:51:00.543479 596 raft_consensus.cc:1261] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.543972 464 raft_consensus.cc:1261] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.543980 1315 consensus_queue.cc:1035] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.544490 1315 consensus_queue.cc:1035] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.546226 326 raft_consensus.cc:1261] T 76ada11863ac4cce802de521b842be2d P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.546619 1247 consensus_queue.cc:1035] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.548151 32665 raft_consensus.cc:1261] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.548595 463 raft_consensus.cc:1261] T 3c8353b8e1494e33a069eb6921b81ba5 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.548957 1365 consensus_queue.cc:1035] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.549264 1365 consensus_queue.cc:1035] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.549665 463 raft_consensus.cc:1261] T 76ada11863ac4cce802de521b842be2d P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.552413 1247 consensus_queue.cc:1035] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 114.589 ms
time per row: 0.0572947 ms
I20240426 09:51:00.612120 32517 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:51:00.617383 308 tablet_service.cc:1467] Processing CreateTablet for tablet f34e4326a7cd44669b9a58324d883911 (DEFAULT_TABLE table=rebalance_test_table_4 [id=48f441a035854069abe9fc7e77655991]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.617425 32645 tablet_service.cc:1467] Processing CreateTablet for tablet f34e4326a7cd44669b9a58324d883911 (DEFAULT_TABLE table=rebalance_test_table_4 [id=48f441a035854069abe9fc7e77655991]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.617583 445 tablet_service.cc:1467] Processing CreateTablet for tablet 6d4c29fc73b94132a0c3cb14153a0108 (DEFAULT_TABLE table=rebalance_test_table_4 [id=48f441a035854069abe9fc7e77655991]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.617700 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f34e4326a7cd44669b9a58324d883911. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.617702 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f34e4326a7cd44669b9a58324d883911. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.617820 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d4c29fc73b94132a0c3cb14153a0108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.618330 444 tablet_service.cc:1467] Processing CreateTablet for tablet d6ad2b31cf884e62898c75d2426d8400 (DEFAULT_TABLE table=rebalance_test_table_4 [id=48f441a035854069abe9fc7e77655991]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.618518 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ad2b31cf884e62898c75d2426d8400. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.620477 576 tablet_service.cc:1467] Processing CreateTablet for tablet f34e4326a7cd44669b9a58324d883911 (DEFAULT_TABLE table=rebalance_test_table_4 [id=48f441a035854069abe9fc7e77655991]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.620596 32645 tablet_service.cc:1467] Processing CreateTablet for tablet 6d4c29fc73b94132a0c3cb14153a0108 (DEFAULT_TABLE table=rebalance_test_table_4 [id=48f441a035854069abe9fc7e77655991]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.620632 308 tablet_service.cc:1467] Processing CreateTablet for tablet 6d4c29fc73b94132a0c3cb14153a0108 (DEFAULT_TABLE table=rebalance_test_table_4 [id=48f441a035854069abe9fc7e77655991]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.620780 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f34e4326a7cd44669b9a58324d883911. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.620801 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d4c29fc73b94132a0c3cb14153a0108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.620645 1455 tablet_bootstrap.cc:492] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:00.620908 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d4c29fc73b94132a0c3cb14153a0108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.621837 1455 tablet_bootstrap.cc:654] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.622655 1453 tablet_bootstrap.cc:492] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:00.622838 1454 tablet_bootstrap.cc:492] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:00.623009 574 tablet_service.cc:1467] Processing CreateTablet for tablet d6ad2b31cf884e62898c75d2426d8400 (DEFAULT_TABLE table=rebalance_test_table_4 [id=48f441a035854069abe9fc7e77655991]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.623242 574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ad2b31cf884e62898c75d2426d8400. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.623478 1453 tablet_bootstrap.cc:654] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.623550 1454 tablet_bootstrap.cc:654] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.624347 301 tablet_service.cc:1467] Processing CreateTablet for tablet d6ad2b31cf884e62898c75d2426d8400 (DEFAULT_TABLE table=rebalance_test_table_4 [id=48f441a035854069abe9fc7e77655991]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:00.624552 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6ad2b31cf884e62898c75d2426d8400. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:00.624635 1453 tablet_bootstrap.cc:492] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:00.624725 1453 ts_tablet_manager.cc:1367] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:00.625167 1453 raft_consensus.cc:348] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:00.625283 1453 raft_consensus.cc:374] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.625314 1453 raft_consensus.cc:729] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.625442 1453 consensus_queue.cc:260] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:00.625711 1453 ts_tablet_manager.cc:1398] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:00.625874 1453 tablet_bootstrap.cc:492] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:00.626463 1457 tablet_bootstrap.cc:492] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:00.626683 1453 tablet_bootstrap.cc:654] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.627271 1457 tablet_bootstrap.cc:654] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.627859 1453 tablet_bootstrap.cc:492] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:00.627943 1453 ts_tablet_manager.cc:1367] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:00.628268 1455 tablet_bootstrap.cc:492] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:00.628368 1455 ts_tablet_manager.cc:1367] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:51:00.628341 1453 raft_consensus.cc:348] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.628451 1453 raft_consensus.cc:374] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.628479 1453 raft_consensus.cc:729] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.628602 1453 consensus_queue.cc:260] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.628821 1455 raft_consensus.cc:348] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:00.628896 1455 raft_consensus.cc:374] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.628916 1455 raft_consensus.cc:729] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.628931 1453 ts_tablet_manager.cc:1398] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.629011 1455 consensus_queue.cc:260] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:00.629211 1454 tablet_bootstrap.cc:492] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:00.629269 1455 ts_tablet_manager.cc:1398] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:00.629304 1454 ts_tablet_manager.cc:1367] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:51:00.629745 1455 tablet_bootstrap.cc:492] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:00.629772 1454 raft_consensus.cc:348] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.629889 1454 raft_consensus.cc:374] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.629923 1454 raft_consensus.cc:729] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.630070 1454 consensus_queue.cc:260] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.630331 1454 ts_tablet_manager.cc:1398] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.630506 1454 tablet_bootstrap.cc:492] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:00.630623 1455 tablet_bootstrap.cc:654] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.631450 1454 tablet_bootstrap.cc:654] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.632280 1457 tablet_bootstrap.cc:492] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:00.632316 1455 tablet_bootstrap.cc:492] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:00.632369 1457 ts_tablet_manager.cc:1367] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:51:00.632375 1455 ts_tablet_manager.cc:1367] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:51:00.632782 1455 raft_consensus.cc:348] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.632782 1457 raft_consensus.cc:348] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:00.632889 1455 raft_consensus.cc:374] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.632887 1457 raft_consensus.cc:374] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.632918 1455 raft_consensus.cc:729] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.632918 1457 raft_consensus.cc:729] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.633034 1457 consensus_queue.cc:260] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:00.633034 1455 consensus_queue.cc:260] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.633270 1457 ts_tablet_manager.cc:1398] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:00.633270 1455 ts_tablet_manager.cc:1398] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.633411 1455 tablet_bootstrap.cc:492] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:00.633425 1457 tablet_bootstrap.cc:492] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:00.634526 1455 tablet_bootstrap.cc:654] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.634706 1457 tablet_bootstrap.cc:654] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:00.635998 1455 tablet_bootstrap.cc:492] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:00.636097 1455 ts_tablet_manager.cc:1367] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:51:00.636454 1457 tablet_bootstrap.cc:492] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:00.636473 1455 raft_consensus.cc:348] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.636523 1457 ts_tablet_manager.cc:1367] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:51:00.636552 1455 raft_consensus.cc:374] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.636580 1455 raft_consensus.cc:729] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.636687 1455 consensus_queue.cc:260] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.636912 1455 ts_tablet_manager.cc:1398] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:00.636904 1457 raft_consensus.cc:348] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.636992 1457 raft_consensus.cc:374] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.637022 1457 raft_consensus.cc:729] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.637364 1454 tablet_bootstrap.cc:492] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:00.637451 1454 ts_tablet_manager.cc:1367] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240426 09:51:00.637152 1457 consensus_queue.cc:260] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.637871 1454 raft_consensus.cc:348] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.637962 1454 raft_consensus.cc:374] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:00.637995 1454 raft_consensus.cc:729] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:00.638111 1454 consensus_queue.cc:260] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.638185 1457 ts_tablet_manager.cc:1398] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:00.638346 1454 ts_tablet_manager.cc:1398] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:00.652341 1315 raft_consensus.cc:482] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:00.652456 1315 raft_consensus.cc:504] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.652843 1315 leader_election.cc:288] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:00.653203 596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad2b31cf884e62898c75d2426d8400" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:00.653209 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad2b31cf884e62898c75d2426d8400" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:00.653331 596 raft_consensus.cc:2454] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:00.653347 464 raft_consensus.cc:2454] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:00.653631 32732 leader_election.cc:302] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:00.653802 1315 raft_consensus.cc:2781] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:00.653849 1315 raft_consensus.cc:482] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:00.653882 1315 raft_consensus.cc:3037] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.654809 1315 raft_consensus.cc:504] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.655185 1315 leader_election.cc:288] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:00.655463 596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad2b31cf884e62898c75d2426d8400" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:00.655473 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad2b31cf884e62898c75d2426d8400" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:00.655566 464 raft_consensus.cc:3037] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.655565 596 raft_consensus.cc:3037] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.656360 596 raft_consensus.cc:2454] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:00.656422 464 raft_consensus.cc:2454] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:00.656687 32729 leader_election.cc:302] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:00.656834 1315 raft_consensus.cc:2781] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:00.656924 1315 raft_consensus.cc:686] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Leader. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:00.657047 1315 consensus_queue.cc:237] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:00.658881 32521 catalog_manager.cc:5484] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130). New cstate: current_term: 1 leader_uuid: "07593a4e1c0242a1b52bb96338b63c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } }
I20240426 09:51:00.667073 1315 raft_consensus.cc:482] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:00.667198 1315 raft_consensus.cc:504] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.667547 1315 leader_election.cc:288] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:00.667941 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4c29fc73b94132a0c3cb14153a0108" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:00.667961 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4c29fc73b94132a0c3cb14153a0108" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:00.668090 464 raft_consensus.cc:2454] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:00.668097 32665 raft_consensus.cc:2454] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:00.668365 32732 leader_election.cc:302] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:51:00.668534 1315 raft_consensus.cc:2781] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:00.668588 1315 raft_consensus.cc:482] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:00.668641 1315 raft_consensus.cc:3037] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.669406 1315 raft_consensus.cc:504] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.669699 1315 leader_election.cc:288] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:00.670045 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4c29fc73b94132a0c3cb14153a0108" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:00.670094 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4c29fc73b94132a0c3cb14153a0108" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:00.670146 464 raft_consensus.cc:3037] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.670210 32665 raft_consensus.cc:3037] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.670894 464 raft_consensus.cc:2454] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:00.670953 32665 raft_consensus.cc:2454] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:00.671160 32732 leader_election.cc:302] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:00.671320 1315 raft_consensus.cc:2781] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:00.671416 1315 raft_consensus.cc:686] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Leader. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:00.671577 1315 consensus_queue.cc:237] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:00.673182 32521 catalog_manager.cc:5484] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130). New cstate: current_term: 1 leader_uuid: "07593a4e1c0242a1b52bb96338b63c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
I20240426 09:51:00.759538 1247 raft_consensus.cc:482] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:00.759685 1247 raft_consensus.cc:504] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:00.760149 1247 leader_election.cc:288] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:00.760629 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34e4326a7cd44669b9a58324d883911" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:00.760632 596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34e4326a7cd44669b9a58324d883911" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:00.760769 326 raft_consensus.cc:2454] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:00.760759 596 raft_consensus.cc:2454] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:00.761067 32598 leader_election.cc:302] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:00.761230 1247 raft_consensus.cc:2781] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:00.761282 1247 raft_consensus.cc:482] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:00.761307 1247 raft_consensus.cc:3037] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.762043 1247 raft_consensus.cc:504] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:00.762341 1247 leader_election.cc:288] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:00.762645 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34e4326a7cd44669b9a58324d883911" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:00.762748 326 raft_consensus.cc:3037] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.762712 596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34e4326a7cd44669b9a58324d883911" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:00.762876 596 raft_consensus.cc:3037] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:00.763617 596 raft_consensus.cc:2454] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:00.763629 326 raft_consensus.cc:2454] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:00.763895 32598 leader_election.cc:302] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:00.764053 1247 raft_consensus.cc:2781] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:00.764137 1247 raft_consensus.cc:686] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:00.764259 1247 consensus_queue.cc:237] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:00.765933 32521 catalog_manager.cc:5484] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
W20240426 09:51:00.894526 1469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:00.894811 1469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:00.897858 1469 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240426 09:51:00.970445 326 raft_consensus.cc:1261] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.970747 596 raft_consensus.cc:1261] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.971119 1199 consensus_queue.cc:1035] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.971217 594 raft_consensus.cc:1261] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.971632 1494 consensus_queue.cc:1035] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.972582 464 raft_consensus.cc:1261] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.972685 463 raft_consensus.cc:1261] T 6d4c29fc73b94132a0c3cb14153a0108 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.973127 32665 raft_consensus.cc:1261] T 6d4c29fc73b94132a0c3cb14153a0108 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:00.974303 1247 consensus_queue.cc:1035] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.978983 1315 consensus_queue.cc:1035] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.979344 1315 consensus_queue.cc:1035] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:00.979593 1315 consensus_queue.cc:1035] T 6d4c29fc73b94132a0c3cb14153a0108 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:01.005738 1500 mvcc.cc:205] Tried to move back new op lower bound from 7021056249728581632 to 7021056248891760640. Current Snapshot: MvccSnapshot[applied={T|T < 7021056249728581632}]
I20240426 09:51:01.020061 1428 mvcc.cc:205] Tried to move back new op lower bound from 7021056249763995648 to 7021056248452521984. Current Snapshot: MvccSnapshot[applied={T|T < 7021056249729019904}]
INSERT report
rows total: 2000
time total: 102.084 ms
time per row: 0.0510421 ms
I20240426 09:51:01.043488 32521 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:51:01.048764 576 tablet_service.cc:1467] Processing CreateTablet for tablet f05415a6ed614f95be718921868fff20 (DEFAULT_TABLE table=rebalance_test_table_5 [id=1c565ed475a5435e84a119fadc990cbf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.048897 32645 tablet_service.cc:1467] Processing CreateTablet for tablet f05415a6ed614f95be718921868fff20 (DEFAULT_TABLE table=rebalance_test_table_5 [id=1c565ed475a5435e84a119fadc990cbf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.049113 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05415a6ed614f95be718921868fff20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.049202 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05415a6ed614f95be718921868fff20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.049239 445 tablet_service.cc:1467] Processing CreateTablet for tablet b9315a8a86f942bdbcb12f96b1c4acec (DEFAULT_TABLE table=rebalance_test_table_5 [id=1c565ed475a5435e84a119fadc990cbf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.049517 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9315a8a86f942bdbcb12f96b1c4acec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.050078 32644 tablet_service.cc:1467] Processing CreateTablet for tablet b9315a8a86f942bdbcb12f96b1c4acec (DEFAULT_TABLE table=rebalance_test_table_5 [id=1c565ed475a5435e84a119fadc990cbf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.050338 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9315a8a86f942bdbcb12f96b1c4acec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.051479 1453 tablet_bootstrap.cc:492] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:01.051626 1454 tablet_bootstrap.cc:492] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:01.051590 444 tablet_service.cc:1467] Processing CreateTablet for tablet f05415a6ed614f95be718921868fff20 (DEFAULT_TABLE table=rebalance_test_table_5 [id=1c565ed475a5435e84a119fadc990cbf]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.051790 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05415a6ed614f95be718921868fff20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.052340 1453 tablet_bootstrap.cc:654] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.052392 1454 tablet_bootstrap.cc:654] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.052947 1457 tablet_bootstrap.cc:492] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:01.053669 1457 tablet_bootstrap.cc:654] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.053793 301 tablet_service.cc:1467] Processing CreateTablet for tablet 786a6056e8164484b976dc64b1aed9ae (DEFAULT_TABLE table=rebalance_test_table_5 [id=1c565ed475a5435e84a119fadc990cbf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.054061 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 786a6056e8164484b976dc64b1aed9ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.054090 1454 tablet_bootstrap.cc:492] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:01.054162 1454 ts_tablet_manager.cc:1367] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:51:01.054500 576 tablet_service.cc:1467] Processing CreateTablet for tablet b9315a8a86f942bdbcb12f96b1c4acec (DEFAULT_TABLE table=rebalance_test_table_5 [id=1c565ed475a5435e84a119fadc990cbf]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.054567 574 tablet_service.cc:1467] Processing CreateTablet for tablet 786a6056e8164484b976dc64b1aed9ae (DEFAULT_TABLE table=rebalance_test_table_5 [id=1c565ed475a5435e84a119fadc990cbf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.054584 1454 raft_consensus.cc:348] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.054688 1454 raft_consensus.cc:374] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.054728 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9315a8a86f942bdbcb12f96b1c4acec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.054719 1454 raft_consensus.cc:729] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.055016 574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 786a6056e8164484b976dc64b1aed9ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.055048 32644 tablet_service.cc:1467] Processing CreateTablet for tablet 786a6056e8164484b976dc64b1aed9ae (DEFAULT_TABLE table=rebalance_test_table_5 [id=1c565ed475a5435e84a119fadc990cbf]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.055244 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 786a6056e8164484b976dc64b1aed9ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.056260 1455 tablet_bootstrap.cc:492] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:01.056895 1454 consensus_queue.cc:260] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.057091 1455 tablet_bootstrap.cc:654] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.057196 1454 ts_tablet_manager.cc:1398] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:51:01.057361 1454 tablet_bootstrap.cc:492] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:01.057952 1453 tablet_bootstrap.cc:492] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:01.058033 1453 ts_tablet_manager.cc:1367] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240426 09:51:01.058234 1457 tablet_bootstrap.cc:492] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:01.058305 1457 ts_tablet_manager.cc:1367] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240426 09:51:01.058434 1454 tablet_bootstrap.cc:654] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.058461 1453 raft_consensus.cc:348] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.058557 1453 raft_consensus.cc:374] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.058598 1453 raft_consensus.cc:729] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.058725 1453 consensus_queue.cc:260] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.058748 1457 raft_consensus.cc:348] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.058863 1457 raft_consensus.cc:374] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.058895 1457 raft_consensus.cc:729] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.059024 1453 ts_tablet_manager.cc:1398] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:01.059019 1457 consensus_queue.cc:260] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.059334 1457 ts_tablet_manager.cc:1398] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:51:01.059501 1457 tablet_bootstrap.cc:492] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:01.059528 1453 tablet_bootstrap.cc:492] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:01.060534 1453 tablet_bootstrap.cc:654] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.060601 1457 tablet_bootstrap.cc:654] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.062728 1454 tablet_bootstrap.cc:492] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:01.062840 1454 ts_tablet_manager.cc:1367] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:51:01.063297 1454 raft_consensus.cc:348] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.063417 1454 raft_consensus.cc:374] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.063445 1454 raft_consensus.cc:729] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.063548 1457 tablet_bootstrap.cc:492] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:01.063540 1454 consensus_queue.cc:260] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.063616 1457 ts_tablet_manager.cc:1367] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:51:01.063766 1454 ts_tablet_manager.cc:1398] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:01.063916 1457 raft_consensus.cc:348] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.064014 1457 raft_consensus.cc:374] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.064051 1457 raft_consensus.cc:729] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.064185 1457 consensus_queue.cc:260] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.064457 1457 ts_tablet_manager.cc:1398] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:01.064615 1455 tablet_bootstrap.cc:492] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:01.064651 1457 tablet_bootstrap.cc:492] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:01.064700 1455 ts_tablet_manager.cc:1367] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:51:01.064831 1453 tablet_bootstrap.cc:492] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:01.064909 1453 ts_tablet_manager.cc:1367] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:51:01.065099 1455 raft_consensus.cc:348] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:01.065217 1455 raft_consensus.cc:374] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.065243 1455 raft_consensus.cc:729] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.065361 1453 raft_consensus.cc:348] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.065356 1455 consensus_queue.cc:260] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:01.065451 1453 raft_consensus.cc:374] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.065483 1453 raft_consensus.cc:729] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.065498 1457 tablet_bootstrap.cc:654] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.065600 1453 consensus_queue.cc:260] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.065675 1455 ts_tablet_manager.cc:1398] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:01.065865 1453 ts_tablet_manager.cc:1398] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:01.066023 1453 tablet_bootstrap.cc:492] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:01.066504 1457 tablet_bootstrap.cc:492] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:01.066588 1457 ts_tablet_manager.cc:1367] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:01.066916 1453 tablet_bootstrap.cc:654] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.066972 1457 raft_consensus.cc:348] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:01.067061 1457 raft_consensus.cc:374] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.067090 1457 raft_consensus.cc:729] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.067209 1457 consensus_queue.cc:260] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:01.067458 1457 ts_tablet_manager.cc:1398] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:01.068866 1453 tablet_bootstrap.cc:492] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:01.068949 1453 ts_tablet_manager.cc:1367] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:51:01.069388 1453 raft_consensus.cc:348] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:01.069530 1453 raft_consensus.cc:374] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.069569 1453 raft_consensus.cc:729] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.069690 1453 consensus_queue.cc:260] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:01.069967 1453 ts_tablet_manager.cc:1398] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:01.108096 1199 raft_consensus.cc:482] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:01.108214 1199 raft_consensus.cc:504] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.108597 1199 leader_election.cc:288] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:01.108911 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:01.108953 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:01.109040 595 raft_consensus.cc:2454] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:01.109071 463 raft_consensus.cc:2454] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:01.109345 32601 leader_election.cc:302] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:01.109530 1199 raft_consensus.cc:2781] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:01.109584 1199 raft_consensus.cc:482] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:01.109611 1199 raft_consensus.cc:3037] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.110402 1199 raft_consensus.cc:504] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.110702 1199 leader_election.cc:288] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:01.111037 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:01.111076 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:01.111155 595 raft_consensus.cc:3037] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.111212 463 raft_consensus.cc:3037] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.112135 595 raft_consensus.cc:2454] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:01.112272 463 raft_consensus.cc:2454] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:01.112409 32598 leader_election.cc:302] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:01.112627 1199 raft_consensus.cc:2781] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:01.112701 1199 raft_consensus.cc:686] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:01.112856 1199 consensus_queue.cc:237] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.114687 32518 catalog_manager.cc:5484] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:01.155608 1494 raft_consensus.cc:482] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:01.155740 1494 raft_consensus.cc:504] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:01.156211 1494 leader_election.cc:288] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:01.156659 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a6056e8164484b976dc64b1aed9ae" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:01.156684 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a6056e8164484b976dc64b1aed9ae" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:01.156802 595 raft_consensus.cc:2454] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:01.156829 32665 raft_consensus.cc:2454] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:01.157117 32729 leader_election.cc:302] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:01.157287 1494 raft_consensus.cc:2781] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:01.157336 1494 raft_consensus.cc:482] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:01.157375 1494 raft_consensus.cc:3037] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.158360 1494 raft_consensus.cc:504] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:01.158725 1494 leader_election.cc:288] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:01.159060 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a6056e8164484b976dc64b1aed9ae" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:01.159057 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a6056e8164484b976dc64b1aed9ae" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:01.159152 595 raft_consensus.cc:3037] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.159180 32665 raft_consensus.cc:3037] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.159910 32665 raft_consensus.cc:2454] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:01.160037 595 raft_consensus.cc:2454] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:01.160182 32732 leader_election.cc:302] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:51:01.160449 1494 raft_consensus.cc:2781] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:01.160540 1494 raft_consensus.cc:686] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Leader. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:01.160677 1494 consensus_queue.cc:237] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:01.162400 32518 catalog_manager.cc:5484] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130). New cstate: current_term: 1 leader_uuid: "07593a4e1c0242a1b52bb96338b63c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:01.360554 1193 raft_consensus.cc:482] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:01.360699 1193 raft_consensus.cc:504] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.361662 1193 leader_election.cc:288] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:01.362192 1197 raft_consensus.cc:482] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:01.362295 1197 raft_consensus.cc:504] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.362663 1197 leader_election.cc:288] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:01.363147 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05415a6ed614f95be718921868fff20" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:01.363147 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05415a6ed614f95be718921868fff20" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:01.363260 463 raft_consensus.cc:2454] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:01.363299 32665 raft_consensus.cc:2454] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:01.363533 532 leader_election.cc:302] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:01.363785 1197 raft_consensus.cc:2781] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:01.363863 1197 raft_consensus.cc:482] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:01.363919 1197 raft_consensus.cc:3037] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.364957 1197 raft_consensus.cc:504] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.365320 1197 leader_election.cc:288] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:01.365620 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05415a6ed614f95be718921868fff20" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:01.365734 32665 raft_consensus.cc:3037] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.365994 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05415a6ed614f95be718921868fff20" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:01.366103 463 raft_consensus.cc:3037] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.366194 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05415a6ed614f95be718921868fff20" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:01.366204 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05415a6ed614f95be718921868fff20" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:01.366336 595 raft_consensus.cc:2379] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4076eb0699724ee78c2f2f0d6354cf9b in current term 1: Already voted for candidate f9655c9ff2e14edfbcd798d88bca0623 in this term.
I20240426 09:51:01.366711 393 leader_election.cc:302] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b; no voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623
I20240426 09:51:01.367013 463 raft_consensus.cc:2454] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:01.367095 1193 raft_consensus.cc:2726] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240426 09:51:01.367292 532 leader_election.cc:302] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:01.367440 1197 raft_consensus.cc:2781] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:01.367528 1197 raft_consensus.cc:686] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:01.367668 1197 consensus_queue.cc:237] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.369210 32518 catalog_manager.cc:5484] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:01.391611 32665 raft_consensus.cc:2454] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:01.517272 1199 consensus_queue.cc:1035] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:01.520622 1495 consensus_queue.cc:1035] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240426 09:51:01.535022 1520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:01.535292 1520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:01.538354 1520 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240426 09:51:01.540669 1494 consensus_queue.cc:1035] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:01.543887 1494 consensus_queue.cc:1035] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Using table 'rebalance_test_table_5'
I20240426 09:51:01.625180 32665 raft_consensus.cc:1261] T f05415a6ed614f95be718921868fff20 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:01.625823 463 raft_consensus.cc:1261] T f05415a6ed614f95be718921868fff20 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:01.626299 1197 consensus_queue.cc:1035] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:01.626606 1197 consensus_queue.cc:1035] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:01.669267 1557 mvcc.cc:205] Tried to move back new op lower bound from 7021056252410634240 to 7021056251363430400. Current Snapshot: MvccSnapshot[applied={T|T < 7021056252410634240}]
INSERT report
rows total: 2000
time total: 119.261 ms
time per row: 0.0596306 ms
I20240426 09:51:01.701910 32518 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:51:01.707237 32644 tablet_service.cc:1467] Processing CreateTablet for tablet 69957fb9470844adaac241bacbb3a34b (DEFAULT_TABLE table=rebalance_test_table_6 [id=0b10c1691f10454eb738bbacb433759c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.707338 301 tablet_service.cc:1467] Processing CreateTablet for tablet 69957fb9470844adaac241bacbb3a34b (DEFAULT_TABLE table=rebalance_test_table_6 [id=0b10c1691f10454eb738bbacb433759c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.707563 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69957fb9470844adaac241bacbb3a34b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.707556 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69957fb9470844adaac241bacbb3a34b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.707505 444 tablet_service.cc:1467] Processing CreateTablet for tablet 69957fb9470844adaac241bacbb3a34b (DEFAULT_TABLE table=rebalance_test_table_6 [id=0b10c1691f10454eb738bbacb433759c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.707794 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69957fb9470844adaac241bacbb3a34b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.708789 307 tablet_service.cc:1467] Processing CreateTablet for tablet 2105fe0e494d48bb9e75d4f55075ac33 (DEFAULT_TABLE table=rebalance_test_table_6 [id=0b10c1691f10454eb738bbacb433759c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.709040 307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2105fe0e494d48bb9e75d4f55075ac33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.710017 308 tablet_service.cc:1467] Processing CreateTablet for tablet be27b88cb26b48b499de234f7c9de3e2 (DEFAULT_TABLE table=rebalance_test_table_6 [id=0b10c1691f10454eb738bbacb433759c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.710232 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be27b88cb26b48b499de234f7c9de3e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.710374 1581 tablet_bootstrap.cc:492] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:01.710528 445 tablet_service.cc:1467] Processing CreateTablet for tablet 2105fe0e494d48bb9e75d4f55075ac33 (DEFAULT_TABLE table=rebalance_test_table_6 [id=0b10c1691f10454eb738bbacb433759c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.710748 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2105fe0e494d48bb9e75d4f55075ac33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.711377 1581 tablet_bootstrap.cc:654] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.711880 574 tablet_service.cc:1467] Processing CreateTablet for tablet be27b88cb26b48b499de234f7c9de3e2 (DEFAULT_TABLE table=rebalance_test_table_6 [id=0b10c1691f10454eb738bbacb433759c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.712152 574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be27b88cb26b48b499de234f7c9de3e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.712632 32644 tablet_service.cc:1467] Processing CreateTablet for tablet 2105fe0e494d48bb9e75d4f55075ac33 (DEFAULT_TABLE table=rebalance_test_table_6 [id=0b10c1691f10454eb738bbacb433759c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.712855 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2105fe0e494d48bb9e75d4f55075ac33. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.712954 444 tablet_service.cc:1467] Processing CreateTablet for tablet be27b88cb26b48b499de234f7c9de3e2 (DEFAULT_TABLE table=rebalance_test_table_6 [id=0b10c1691f10454eb738bbacb433759c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:01.713173 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be27b88cb26b48b499de234f7c9de3e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:01.714360 1583 tablet_bootstrap.cc:492] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:01.715234 1580 tablet_bootstrap.cc:492] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:01.715466 1583 tablet_bootstrap.cc:654] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.716110 1580 tablet_bootstrap.cc:654] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.716126 1579 tablet_bootstrap.cc:492] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:01.716452 1581 tablet_bootstrap.cc:492] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:01.716548 1581 ts_tablet_manager.cc:1367] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240426 09:51:01.716981 1579 tablet_bootstrap.cc:654] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.716960 1581 raft_consensus.cc:348] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.717077 1581 raft_consensus.cc:374] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.717109 1581 raft_consensus.cc:729] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.717229 1581 consensus_queue.cc:260] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.717476 1581 ts_tablet_manager.cc:1398] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:01.717629 1581 tablet_bootstrap.cc:492] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:01.718175 1583 tablet_bootstrap.cc:492] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:01.718255 1583 ts_tablet_manager.cc:1367] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:51:01.718345 1581 tablet_bootstrap.cc:654] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.718688 1583 raft_consensus.cc:348] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.718865 1583 raft_consensus.cc:374] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.718910 1583 raft_consensus.cc:729] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.719050 1583 consensus_queue.cc:260] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.719383 1583 ts_tablet_manager.cc:1398] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:01.719816 1581 tablet_bootstrap.cc:492] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:01.719897 1581 ts_tablet_manager.cc:1367] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:01.719940 1579 tablet_bootstrap.cc:492] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:01.720090 1579 ts_tablet_manager.cc:1367] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:51:01.720248 1581 raft_consensus.cc:348] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:01.720327 1581 raft_consensus.cc:374] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.720363 1581 raft_consensus.cc:729] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.720485 1581 consensus_queue.cc:260] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:01.720482 1579 raft_consensus.cc:348] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.720587 1579 raft_consensus.cc:374] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.720629 1579 raft_consensus.cc:729] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.720733 1581 ts_tablet_manager.cc:1398] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:01.720767 1579 consensus_queue.cc:260] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.720889 1581 tablet_bootstrap.cc:492] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:01.721025 1579 ts_tablet_manager.cc:1398] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:01.721166 1579 tablet_bootstrap.cc:492] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:01.721947 1581 tablet_bootstrap.cc:654] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.722030 1579 tablet_bootstrap.cc:654] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.723170 1581 tablet_bootstrap.cc:492] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:01.723294 1579 tablet_bootstrap.cc:492] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:01.723290 1581 ts_tablet_manager.cc:1367] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:51:01.723381 1579 ts_tablet_manager.cc:1367] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:01.723753 1579 raft_consensus.cc:348] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:01.723846 1579 raft_consensus.cc:374] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.723875 1579 raft_consensus.cc:729] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.723968 1579 consensus_queue.cc:260] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:01.724195 1579 ts_tablet_manager.cc:1398] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:01.724330 1581 raft_consensus.cc:348] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.724438 1581 raft_consensus.cc:374] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.724473 1581 raft_consensus.cc:729] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.724596 1581 consensus_queue.cc:260] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.724821 1581 ts_tablet_manager.cc:1398] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:01.735200 1580 tablet_bootstrap.cc:492] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:01.735318 1580 ts_tablet_manager.cc:1367] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240426 09:51:01.735721 1580 raft_consensus.cc:348] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.735826 1580 raft_consensus.cc:374] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.735862 1580 raft_consensus.cc:729] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.736001 1580 consensus_queue.cc:260] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.736260 1580 ts_tablet_manager.cc:1398] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:01.736414 1580 tablet_bootstrap.cc:492] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:01.737169 1580 tablet_bootstrap.cc:654] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.738325 1580 tablet_bootstrap.cc:492] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:01.738401 1580 ts_tablet_manager.cc:1367] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:01.738730 1580 raft_consensus.cc:348] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:01.738854 1580 raft_consensus.cc:374] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.738900 1580 raft_consensus.cc:729] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.739018 1580 consensus_queue.cc:260] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:01.739259 1580 ts_tablet_manager.cc:1398] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:01.739434 1580 tablet_bootstrap.cc:492] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:01.740319 1580 tablet_bootstrap.cc:654] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:01.741348 1580 tablet_bootstrap.cc:492] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:01.741423 1580 ts_tablet_manager.cc:1367] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:01.741869 1580 raft_consensus.cc:348] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.741974 1580 raft_consensus.cc:374] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:01.742009 1580 raft_consensus.cc:729] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:01.742136 1580 consensus_queue.cc:260] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.742388 1580 ts_tablet_manager.cc:1398] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:01.852516 1193 raft_consensus.cc:482] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:01.852664 1193 raft_consensus.cc:504] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.853139 1193 leader_election.cc:288] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:01.853483 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69957fb9470844adaac241bacbb3a34b" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:01.853622 32665 raft_consensus.cc:2454] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4076eb0699724ee78c2f2f0d6354cf9b in term 0.
I20240426 09:51:01.853930 396 leader_election.cc:302] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:01.854103 1193 raft_consensus.cc:2781] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:01.854156 1193 raft_consensus.cc:482] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:01.854200 1193 raft_consensus.cc:3037] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.855103 1193 raft_consensus.cc:504] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.855392 1193 leader_election.cc:288] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:01.855697 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69957fb9470844adaac241bacbb3a34b" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:01.855811 32665 raft_consensus.cc:3037] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.856601 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69957fb9470844adaac241bacbb3a34b" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:01.856681 32665 raft_consensus.cc:2454] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4076eb0699724ee78c2f2f0d6354cf9b in term 1.
I20240426 09:51:01.856660 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69957fb9470844adaac241bacbb3a34b" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:01.856730 327 raft_consensus.cc:2454] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4076eb0699724ee78c2f2f0d6354cf9b in term 0.
I20240426 09:51:01.856961 396 leader_election.cc:302] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:01.857122 1193 raft_consensus.cc:2781] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:01.857345 1193 raft_consensus.cc:686] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 LEADER]: Becoming Leader. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Running, Role: LEADER
I20240426 09:51:01.857699 1193 consensus_queue.cc:237] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.859879 32514 catalog_manager.cc:5484] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131). New cstate: current_term: 1 leader_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:01.929687 1576 raft_consensus.cc:482] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:01.929811 1576 raft_consensus.cc:504] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.930208 1576 leader_election.cc:288] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:01.930572 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be27b88cb26b48b499de234f7c9de3e2" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:01.930608 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be27b88cb26b48b499de234f7c9de3e2" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:01.930709 326 raft_consensus.cc:2454] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:01.930755 464 raft_consensus.cc:2454] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:01.931005 532 leader_election.cc:302] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:01.931350 1576 raft_consensus.cc:2781] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:01.931406 1576 raft_consensus.cc:482] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:01.931442 1576 raft_consensus.cc:3037] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.932248 1576 raft_consensus.cc:504] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.932552 1576 leader_election.cc:288] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:01.932852 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be27b88cb26b48b499de234f7c9de3e2" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:01.932930 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be27b88cb26b48b499de234f7c9de3e2" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:01.932986 326 raft_consensus.cc:3037] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.933046 464 raft_consensus.cc:3037] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.933951 326 raft_consensus.cc:2454] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:01.933951 464 raft_consensus.cc:2454] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:01.934217 532 leader_election.cc:302] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:01.934378 1576 raft_consensus.cc:2781] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:01.934459 1576 raft_consensus.cc:686] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:01.934592 1576 consensus_queue.cc:237] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:01.936453 32514 catalog_manager.cc:5484] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:01.984647 1315 raft_consensus.cc:482] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:01.984766 1315 raft_consensus.cc:504] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:01.985113 1315 leader_election.cc:288] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:01.985453 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2105fe0e494d48bb9e75d4f55075ac33" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:01.985509 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2105fe0e494d48bb9e75d4f55075ac33" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:01.985600 464 raft_consensus.cc:2454] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:01.985630 32665 raft_consensus.cc:2454] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:01.985872 32732 leader_election.cc:302] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:01.986013 1315 raft_consensus.cc:2781] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:01.986088 1315 raft_consensus.cc:482] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:01.986120 1315 raft_consensus.cc:3037] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.987135 1315 raft_consensus.cc:504] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:01.987426 1315 leader_election.cc:288] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:01.987699 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2105fe0e494d48bb9e75d4f55075ac33" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:01.987774 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2105fe0e494d48bb9e75d4f55075ac33" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:01.987825 464 raft_consensus.cc:3037] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.987875 32665 raft_consensus.cc:3037] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:01.988572 464 raft_consensus.cc:2454] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:01.988677 32665 raft_consensus.cc:2454] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:01.988960 32732 leader_election.cc:302] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:01.989358 1315 raft_consensus.cc:2781] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:01.989460 1315 raft_consensus.cc:686] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Leader. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:01.989619 1315 consensus_queue.cc:237] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:01.991283 32514 catalog_manager.cc:5484] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130). New cstate: current_term: 1 leader_uuid: "07593a4e1c0242a1b52bb96338b63c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
W20240426 09:51:02.111608 1597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:02.111850 1597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:02.114908 1597 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240426 09:51:02.221688 326 raft_consensus.cc:3037] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.221822 32665 raft_consensus.cc:1261] T 69957fb9470844adaac241bacbb3a34b P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer 4076eb0699724ee78c2f2f0d6354cf9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.222246 1193 consensus_queue.cc:1035] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.222743 326 raft_consensus.cc:1261] T 69957fb9470844adaac241bacbb3a34b P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer 4076eb0699724ee78c2f2f0d6354cf9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.223109 1193 consensus_queue.cc:1035] T 69957fb9470844adaac241bacbb3a34b P 4076eb0699724ee78c2f2f0d6354cf9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.224908 464 raft_consensus.cc:1261] T be27b88cb26b48b499de234f7c9de3e2 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.225203 327 raft_consensus.cc:1261] T be27b88cb26b48b499de234f7c9de3e2 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.225855 1576 consensus_queue.cc:1035] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.226132 1576 consensus_queue.cc:1035] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.228014 32664 raft_consensus.cc:1261] T 2105fe0e494d48bb9e75d4f55075ac33 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.228427 1494 consensus_queue.cc:1035] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.230188 463 raft_consensus.cc:1261] T 2105fe0e494d48bb9e75d4f55075ac33 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.230577 1494 consensus_queue.cc:1035] T 2105fe0e494d48bb9e75d4f55075ac33 P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.240774 1629 mvcc.cc:205] Tried to move back new op lower bound from 7021056254901002240 to 7021056253685551104. Current Snapshot: MvccSnapshot[applied={T|T < 7021056254869045248}]
INSERT report
rows total: 2000
time total: 128.761 ms
time per row: 0.0643803 ms
I20240426 09:51:02.286185 32520 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:51:02.291236 308 tablet_service.cc:1467] Processing CreateTablet for tablet 2eae2c486afa483ab3ed754cc720ed8f (DEFAULT_TABLE table=rebalance_test_table_7 [id=bbd1d7434aef4a8b873f73078c0ae887]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.291249 574 tablet_service.cc:1467] Processing CreateTablet for tablet 2eae2c486afa483ab3ed754cc720ed8f (DEFAULT_TABLE table=rebalance_test_table_7 [id=bbd1d7434aef4a8b873f73078c0ae887]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.291534 574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eae2c486afa483ab3ed754cc720ed8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.291533 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eae2c486afa483ab3ed754cc720ed8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.293964 32644 tablet_service.cc:1467] Processing CreateTablet for tablet 2eae2c486afa483ab3ed754cc720ed8f (DEFAULT_TABLE table=rebalance_test_table_7 [id=bbd1d7434aef4a8b873f73078c0ae887]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.294060 445 tablet_service.cc:1467] Processing CreateTablet for tablet dff34820cccc4ba2b7213724d8969ddb (DEFAULT_TABLE table=rebalance_test_table_7 [id=bbd1d7434aef4a8b873f73078c0ae887]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.294247 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eae2c486afa483ab3ed754cc720ed8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.294291 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff34820cccc4ba2b7213724d8969ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.294261 576 tablet_service.cc:1467] Processing CreateTablet for tablet dff34820cccc4ba2b7213724d8969ddb (DEFAULT_TABLE table=rebalance_test_table_7 [id=bbd1d7434aef4a8b873f73078c0ae887]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.294061 444 tablet_service.cc:1467] Processing CreateTablet for tablet c4fbe777c8f94d80b3ae5df7bef3b724 (DEFAULT_TABLE table=rebalance_test_table_7 [id=bbd1d7434aef4a8b873f73078c0ae887]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.294492 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff34820cccc4ba2b7213724d8969ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.294507 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4fbe777c8f94d80b3ae5df7bef3b724. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.295017 32645 tablet_service.cc:1467] Processing CreateTablet for tablet c4fbe777c8f94d80b3ae5df7bef3b724 (DEFAULT_TABLE table=rebalance_test_table_7 [id=bbd1d7434aef4a8b873f73078c0ae887]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.295219 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4fbe777c8f94d80b3ae5df7bef3b724. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.296628 574 tablet_service.cc:1467] Processing CreateTablet for tablet c4fbe777c8f94d80b3ae5df7bef3b724 (DEFAULT_TABLE table=rebalance_test_table_7 [id=bbd1d7434aef4a8b873f73078c0ae887]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.296836 574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4fbe777c8f94d80b3ae5df7bef3b724. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.297585 308 tablet_service.cc:1467] Processing CreateTablet for tablet dff34820cccc4ba2b7213724d8969ddb (DEFAULT_TABLE table=rebalance_test_table_7 [id=bbd1d7434aef4a8b873f73078c0ae887]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.297641 1663 tablet_bootstrap.cc:492] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:02.297816 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff34820cccc4ba2b7213724d8969ddb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.298507 1663 tablet_bootstrap.cc:654] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.298841 1661 tablet_bootstrap.cc:492] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:02.299533 1663 tablet_bootstrap.cc:492] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:02.299624 1663 ts_tablet_manager.cc:1367] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:51:02.299703 1661 tablet_bootstrap.cc:654] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.299724 1664 tablet_bootstrap.cc:492] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:02.300077 1663 raft_consensus.cc:348] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:02.300184 1663 raft_consensus.cc:374] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.300217 1663 raft_consensus.cc:729] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.300343 1663 consensus_queue.cc:260] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:02.300479 1662 tablet_bootstrap.cc:492] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:02.300617 1664 tablet_bootstrap.cc:654] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.300630 1663 ts_tablet_manager.cc:1398] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:02.300777 1663 tablet_bootstrap.cc:492] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:02.301375 1662 tablet_bootstrap.cc:654] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.301601 1663 tablet_bootstrap.cc:654] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.301704 1664 tablet_bootstrap.cc:492] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:02.301780 1664 ts_tablet_manager.cc:1367] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:51:02.302271 1664 raft_consensus.cc:348] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.302343 1661 tablet_bootstrap.cc:492] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:02.302368 1664 raft_consensus.cc:374] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.302415 1661 ts_tablet_manager.cc:1367] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:51:02.302424 1664 raft_consensus.cc:729] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.302554 1664 consensus_queue.cc:260] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.302891 1664 ts_tablet_manager.cc:1398] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:02.302853 1661 raft_consensus.cc:348] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:02.302944 1661 raft_consensus.cc:374] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.302976 1661 raft_consensus.cc:729] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.303006 1663 tablet_bootstrap.cc:492] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:02.303057 1664 tablet_bootstrap.cc:492] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:02.303081 1663 ts_tablet_manager.cc:1367] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:02.303112 1661 consensus_queue.cc:260] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:02.303439 1661 ts_tablet_manager.cc:1398] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:02.303580 1661 tablet_bootstrap.cc:492] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:02.303532 1663 raft_consensus.cc:348] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.303634 1663 raft_consensus.cc:374] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.303669 1663 raft_consensus.cc:729] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.303766 1663 consensus_queue.cc:260] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.303910 1664 tablet_bootstrap.cc:654] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.304008 1663 ts_tablet_manager.cc:1398] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:51:02.304477 1661 tablet_bootstrap.cc:654] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.305446 1495 raft_consensus.cc:482] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:02.305542 1661 tablet_bootstrap.cc:492] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:02.305521 1495 raft_consensus.cc:504] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:02.305619 1661 ts_tablet_manager.cc:1367] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:02.305902 1495 leader_election.cc:288] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:02.306015 1661 raft_consensus.cc:348] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.306108 1661 raft_consensus.cc:374] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.306141 1661 raft_consensus.cc:729] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.306278 1661 consensus_queue.cc:260] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.306376 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:02.306564 1661 ts_tablet_manager.cc:1398] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240426 09:51:02.306888 32598 leader_election.cc:341] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:51:02.306921 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:02.307029 326 raft_consensus.cc:2454] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:02.307304 32601 leader_election.cc:302] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters: f9655c9ff2e14edfbcd798d88bca0623
I20240426 09:51:02.307535 1495 raft_consensus.cc:2781] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:02.307595 1495 raft_consensus.cc:482] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:02.307636 1495 raft_consensus.cc:3037] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.307971 1664 tablet_bootstrap.cc:492] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:02.308024 1664 ts_tablet_manager.cc:1367] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240426 09:51:02.308342 1664 raft_consensus.cc:348] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.308429 1664 raft_consensus.cc:374] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.308459 1664 raft_consensus.cc:729] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.308488 1495 raft_consensus.cc:504] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:02.308571 1664 consensus_queue.cc:260] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.308787 1495 leader_election.cc:288] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:02.309077 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:02.309175 326 raft_consensus.cc:3037] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.309289 1664 ts_tablet_manager.cc:1398] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:02.309299 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
W20240426 09:51:02.309701 32598 leader_election.cc:341] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661): Illegal state: must be running to vote when last-logged opid is not known
I20240426 09:51:02.310096 326 raft_consensus.cc:2454] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:02.310417 32601 leader_election.cc:302] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters: f9655c9ff2e14edfbcd798d88bca0623
I20240426 09:51:02.310618 1495 raft_consensus.cc:2781] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:02.310706 1495 raft_consensus.cc:686] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:02.310845 1662 tablet_bootstrap.cc:492] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:02.310830 1495 consensus_queue.cc:237] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:02.310938 1662 ts_tablet_manager.cc:1367] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240426 09:51:02.311363 1662 raft_consensus.cc:348] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:02.311452 1662 raft_consensus.cc:374] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.311473 1662 raft_consensus.cc:729] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.311571 1662 consensus_queue.cc:260] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:02.311862 1662 ts_tablet_manager.cc:1398] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:02.312093 1662 tablet_bootstrap.cc:492] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:02.312865 1662 tablet_bootstrap.cc:654] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.313614 32520 catalog_manager.cc:5484] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
I20240426 09:51:02.314081 1662 tablet_bootstrap.cc:492] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:02.314152 1662 ts_tablet_manager.cc:1367] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:02.314550 1662 raft_consensus.cc:348] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.314661 1662 raft_consensus.cc:374] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.314694 1662 raft_consensus.cc:729] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.314821 1662 consensus_queue.cc:260] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.315146 1662 ts_tablet_manager.cc:1398] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:02.315310 1662 tablet_bootstrap.cc:492] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:02.316196 1662 tablet_bootstrap.cc:654] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.317088 1662 tablet_bootstrap.cc:492] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:02.317173 1662 ts_tablet_manager.cc:1367] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:02.317524 1662 raft_consensus.cc:348] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.317596 1662 raft_consensus.cc:374] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.317621 1662 raft_consensus.cc:729] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.317749 1662 consensus_queue.cc:260] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.317996 1662 ts_tablet_manager.cc:1398] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:02.560717 1592 raft_consensus.cc:482] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:02.560824 1592 raft_consensus.cc:504] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.561162 1592 leader_election.cc:288] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:02.561491 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:02.561522 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:02.561627 32665 raft_consensus.cc:2454] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4076eb0699724ee78c2f2f0d6354cf9b in term 0.
I20240426 09:51:02.561621 595 raft_consensus.cc:2454] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4076eb0699724ee78c2f2f0d6354cf9b in term 0.
I20240426 09:51:02.561889 393 leader_election.cc:302] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:02.562028 1592 raft_consensus.cc:2781] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:02.562080 1592 raft_consensus.cc:482] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:02.562121 1592 raft_consensus.cc:3037] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.562922 1592 raft_consensus.cc:504] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.563210 1592 leader_election.cc:288] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:02.563455 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:02.563555 595 raft_consensus.cc:3037] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.563572 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724" candidate_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:02.563684 32665 raft_consensus.cc:3037] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.564254 595 raft_consensus.cc:2454] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4076eb0699724ee78c2f2f0d6354cf9b in term 1.
I20240426 09:51:02.564425 32665 raft_consensus.cc:2454] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4076eb0699724ee78c2f2f0d6354cf9b in term 1.
I20240426 09:51:02.564532 393 leader_election.cc:302] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:02.564713 1592 raft_consensus.cc:2781] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:02.564801 1592 raft_consensus.cc:686] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 LEADER]: Becoming Leader. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Running, Role: LEADER
I20240426 09:51:02.564936 1592 consensus_queue.cc:237] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.566505 32523 catalog_manager.cc:5484] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131). New cstate: current_term: 1 leader_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:02.574908 1649 raft_consensus.cc:482] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:02.575028 1649 raft_consensus.cc:504] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.575387 1649 leader_election.cc:288] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:02.575736 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff34820cccc4ba2b7213724d8969ddb" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:02.575778 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff34820cccc4ba2b7213724d8969ddb" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:02.575855 326 raft_consensus.cc:2454] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:02.575907 463 raft_consensus.cc:2454] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:02.576154 532 leader_election.cc:302] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:02.576320 1649 raft_consensus.cc:2781] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:02.576373 1649 raft_consensus.cc:482] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:02.576427 1649 raft_consensus.cc:3037] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.577247 1649 raft_consensus.cc:504] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.577517 1649 leader_election.cc:288] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:02.577785 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff34820cccc4ba2b7213724d8969ddb" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:02.577842 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff34820cccc4ba2b7213724d8969ddb" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:02.577891 326 raft_consensus.cc:3037] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.577958 463 raft_consensus.cc:3037] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.578560 326 raft_consensus.cc:2454] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:02.578629 463 raft_consensus.cc:2454] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:02.578828 532 leader_election.cc:302] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:02.578970 1649 raft_consensus.cc:2781] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:02.579051 1649 raft_consensus.cc:686] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:02.579178 1649 consensus_queue.cc:237] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.580837 32523 catalog_manager.cc:5484] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
W20240426 09:51:02.690244 1677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:02.690488 1677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:02.693357 1677 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240426 09:51:02.739892 595 raft_consensus.cc:3037] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.741497 1495 consensus_queue.cc:1035] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.744904 1495 consensus_queue.cc:1035] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.770553 32665 raft_consensus.cc:1261] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer 4076eb0699724ee78c2f2f0d6354cf9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.771126 595 raft_consensus.cc:1261] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Refusing update from remote peer 4076eb0699724ee78c2f2f0d6354cf9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.771253 1592 consensus_queue.cc:1035] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.775918 1193 consensus_queue.cc:1035] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.778311 464 raft_consensus.cc:1261] T dff34820cccc4ba2b7213724d8969ddb P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.778900 1576 consensus_queue.cc:1035] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:02.779446 326 raft_consensus.cc:1261] T dff34820cccc4ba2b7213724d8969ddb P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:02.780459 1576 consensus_queue.cc:1035] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 110.8 ms
time per row: 0.0553998 ms
I20240426 09:51:02.847122 32518 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:51:02.852336 308 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8ebf7b59324fcc868332f67b6fef1d (DEFAULT_TABLE table=rebalance_test_table_8 [id=466d5eca54fe4676a3272fd9b9fea2bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.852380 32645 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8ebf7b59324fcc868332f67b6fef1d (DEFAULT_TABLE table=rebalance_test_table_8 [id=466d5eca54fe4676a3272fd9b9fea2bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.852654 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8ebf7b59324fcc868332f67b6fef1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.852655 32645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8ebf7b59324fcc868332f67b6fef1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.853122 574 tablet_service.cc:1467] Processing CreateTablet for tablet 0b8ebf7b59324fcc868332f67b6fef1d (DEFAULT_TABLE table=rebalance_test_table_8 [id=466d5eca54fe4676a3272fd9b9fea2bb]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.853401 574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b8ebf7b59324fcc868332f67b6fef1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.853410 445 tablet_service.cc:1467] Processing CreateTablet for tablet 739229b67c4d49e7970c6cd9eb47690e (DEFAULT_TABLE table=rebalance_test_table_8 [id=466d5eca54fe4676a3272fd9b9fea2bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.853648 445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739229b67c4d49e7970c6cd9eb47690e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.855034 576 tablet_service.cc:1467] Processing CreateTablet for tablet 18d93e5032a04e069ff7a924078a9989 (DEFAULT_TABLE table=rebalance_test_table_8 [id=466d5eca54fe4676a3272fd9b9fea2bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.855069 444 tablet_service.cc:1467] Processing CreateTablet for tablet 18d93e5032a04e069ff7a924078a9989 (DEFAULT_TABLE table=rebalance_test_table_8 [id=466d5eca54fe4676a3272fd9b9fea2bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.855254 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18d93e5032a04e069ff7a924078a9989. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.855297 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18d93e5032a04e069ff7a924078a9989. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.855823 1732 tablet_bootstrap.cc:492] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:02.856218 32644 tablet_service.cc:1467] Processing CreateTablet for tablet 739229b67c4d49e7970c6cd9eb47690e (DEFAULT_TABLE table=rebalance_test_table_8 [id=466d5eca54fe4676a3272fd9b9fea2bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.856460 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739229b67c4d49e7970c6cd9eb47690e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.856817 1732 tablet_bootstrap.cc:654] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.857669 1731 tablet_bootstrap.cc:492] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:02.857656 308 tablet_service.cc:1467] Processing CreateTablet for tablet 739229b67c4d49e7970c6cd9eb47690e (DEFAULT_TABLE table=rebalance_test_table_8 [id=466d5eca54fe4676a3272fd9b9fea2bb]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.857896 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 739229b67c4d49e7970c6cd9eb47690e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.858561 32643 tablet_service.cc:1467] Processing CreateTablet for tablet 18d93e5032a04e069ff7a924078a9989 (DEFAULT_TABLE table=rebalance_test_table_8 [id=466d5eca54fe4676a3272fd9b9fea2bb]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:02.858814 32643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18d93e5032a04e069ff7a924078a9989. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:02.858903 1733 tablet_bootstrap.cc:492] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:02.858899 1731 tablet_bootstrap.cc:654] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.859534 1730 tablet_bootstrap.cc:492] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:02.859800 1733 tablet_bootstrap.cc:654] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.860385 1730 tablet_bootstrap.cc:654] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.861205 1733 tablet_bootstrap.cc:492] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:02.861224 1731 tablet_bootstrap.cc:492] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:02.861306 1731 ts_tablet_manager.cc:1367] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240426 09:51:02.861418 1733 ts_tablet_manager.cc:1367] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240426 09:51:02.861760 1731 raft_consensus.cc:348] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.861953 1731 raft_consensus.cc:374] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.861932 1733 raft_consensus.cc:348] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.862042 1731 raft_consensus.cc:729] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.862094 1732 tablet_bootstrap.cc:492] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:02.862108 1733 raft_consensus.cc:374] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.862149 1733 raft_consensus.cc:729] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.862179 1732 ts_tablet_manager.cc:1367] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:51:02.862282 1730 tablet_bootstrap.cc:492] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:02.862244 1731 consensus_queue.cc:260] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.862298 1733 consensus_queue.cc:260] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.862360 1730 ts_tablet_manager.cc:1367] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240426 09:51:02.862609 1731 ts_tablet_manager.cc:1398] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:02.862629 1732 raft_consensus.cc:348] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:02.862721 1732 raft_consensus.cc:374] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.862753 1732 raft_consensus.cc:729] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.862830 1731 tablet_bootstrap.cc:492] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:02.862802 1730 raft_consensus.cc:348] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.862898 1730 raft_consensus.cc:374] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.862928 1730 raft_consensus.cc:729] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.862912 1732 consensus_queue.cc:260] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:02.863041 1730 consensus_queue.cc:260] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.863186 1732 ts_tablet_manager.cc:1398] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:51:02.863205 1733 ts_tablet_manager.cc:1398] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:51:02.863296 1730 ts_tablet_manager.cc:1398] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:51:02.863361 1732 tablet_bootstrap.cc:492] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:02.863440 1730 tablet_bootstrap.cc:492] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:02.863808 1731 tablet_bootstrap.cc:654] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.864161 1733 tablet_bootstrap.cc:492] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:02.864223 1732 tablet_bootstrap.cc:654] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.864312 1730 tablet_bootstrap.cc:654] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.864828 1731 tablet_bootstrap.cc:492] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:02.864894 1731 ts_tablet_manager.cc:1367] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:51:02.865239 1733 tablet_bootstrap.cc:654] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.865267 1731 raft_consensus.cc:348] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:02.865362 1731 raft_consensus.cc:374] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.865392 1731 raft_consensus.cc:729] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.865515 1731 consensus_queue.cc:260] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:02.865790 1731 ts_tablet_manager.cc:1398] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:02.865955 1731 tablet_bootstrap.cc:492] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:02.866907 1731 tablet_bootstrap.cc:654] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:02.868039 1732 tablet_bootstrap.cc:492] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:02.868127 1732 ts_tablet_manager.cc:1367] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240426 09:51:02.868561 1733 tablet_bootstrap.cc:492] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:02.868551 1732 raft_consensus.cc:348] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.868664 1732 raft_consensus.cc:374] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.868669 1733 ts_tablet_manager.cc:1367] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240426 09:51:02.868700 1732 raft_consensus.cc:729] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.868817 1732 consensus_queue.cc:260] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.869091 1732 ts_tablet_manager.cc:1398] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:51:02.869127 1733 raft_consensus.cc:348] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.869217 1733 raft_consensus.cc:374] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.869246 1733 raft_consensus.cc:729] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.869347 1733 consensus_queue.cc:260] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.869629 1733 ts_tablet_manager.cc:1398] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:51:02.869944 1731 tablet_bootstrap.cc:492] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:02.870028 1731 ts_tablet_manager.cc:1367] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240426 09:51:02.870424 1731 raft_consensus.cc:348] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.870548 1731 raft_consensus.cc:374] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.870582 1731 raft_consensus.cc:729] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.870702 1731 consensus_queue.cc:260] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.870955 1731 ts_tablet_manager.cc:1398] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:02.870970 1730 tablet_bootstrap.cc:492] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:02.871044 1730 ts_tablet_manager.cc:1367] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20240426 09:51:02.871450 1730 raft_consensus.cc:348] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:02.871557 1730 raft_consensus.cc:374] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:02.871609 1730 raft_consensus.cc:729] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:02.871728 1730 consensus_queue.cc:260] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:02.872027 1730 ts_tablet_manager.cc:1398] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:02.978338 1725 raft_consensus.cc:482] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:02.978479 1725 raft_consensus.cc:504] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.978993 1725 leader_election.cc:288] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:02.979420 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:02.979441 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:02.979547 464 raft_consensus.cc:2454] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:02.979579 594 raft_consensus.cc:2454] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:02.979835 32598 leader_election.cc:302] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:02.980012 1725 raft_consensus.cc:2781] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:02.980079 1725 raft_consensus.cc:482] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:02.980114 1725 raft_consensus.cc:3037] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.981153 1725 raft_consensus.cc:504] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.981477 1725 leader_election.cc:288] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:02.981778 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:02.981853 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:02.981892 594 raft_consensus.cc:3037] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.981968 464 raft_consensus.cc:3037] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:02.982683 464 raft_consensus.cc:2454] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:02.982825 594 raft_consensus.cc:2454] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:02.983031 32601 leader_election.cc:302] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:02.983194 1725 raft_consensus.cc:2781] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:02.983278 1725 raft_consensus.cc:686] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:02.983408 1725 consensus_queue.cc:237] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:02.985203 32522 catalog_manager.cc:5484] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:02.998627 1576 raft_consensus.cc:482] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:02.998724 1576 raft_consensus.cc:504] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:02.999137 1576 leader_election.cc:288] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:02.999444 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ebf7b59324fcc868332f67b6fef1d" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:02.999526 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ebf7b59324fcc868332f67b6fef1d" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:02.999593 326 raft_consensus.cc:2454] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:02.999672 32664 raft_consensus.cc:2454] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:02.999964 532 leader_election.cc:302] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:03.000188 1576 raft_consensus.cc:2781] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:03.000236 1576 raft_consensus.cc:482] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:03.000279 1576 raft_consensus.cc:3037] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.001121 1576 raft_consensus.cc:504] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.001410 1576 leader_election.cc:288] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:03.001717 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ebf7b59324fcc868332f67b6fef1d" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:03.001777 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ebf7b59324fcc868332f67b6fef1d" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:03.001849 326 raft_consensus.cc:3037] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.001892 32664 raft_consensus.cc:3037] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.002645 32664 raft_consensus.cc:2454] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:03.002748 326 raft_consensus.cc:2454] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:03.002976 532 leader_election.cc:302] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:03.003288 1576 raft_consensus.cc:2781] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:03.003489 1576 raft_consensus.cc:686] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:03.003657 1576 consensus_queue.cc:237] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.005429 32522 catalog_manager.cc:5484] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:03.202682 1315 raft_consensus.cc:482] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:03.202826 1315 raft_consensus.cc:504] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:03.203184 1315 leader_election.cc:288] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:03.203716 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739229b67c4d49e7970c6cd9eb47690e" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:03.203778 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739229b67c4d49e7970c6cd9eb47690e" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2" is_pre_election: true
I20240426 09:51:03.203869 464 raft_consensus.cc:2454] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:03.203974 32664 raft_consensus.cc:2454] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 0.
I20240426 09:51:03.204156 32732 leader_election.cc:302] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:03.204419 1315 raft_consensus.cc:2781] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:03.204468 1315 raft_consensus.cc:482] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:03.204493 1315 raft_consensus.cc:3037] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.205229 1315 raft_consensus.cc:504] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:03.205488 1315 leader_election.cc:288] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:03.205825 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739229b67c4d49e7970c6cd9eb47690e" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:03.205842 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739229b67c4d49e7970c6cd9eb47690e" candidate_uuid: "07593a4e1c0242a1b52bb96338b63c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:03.205922 464 raft_consensus.cc:3037] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.205960 32664 raft_consensus.cc:3037] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.206641 464 raft_consensus.cc:2454] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:03.206696 32664 raft_consensus.cc:2454] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07593a4e1c0242a1b52bb96338b63c8c in term 1.
I20240426 09:51:03.207011 32732 leader_election.cc:302] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:51:03.207165 1315 raft_consensus.cc:2781] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:03.207278 1315 raft_consensus.cc:686] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Leader. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:03.207414 1315 consensus_queue.cc:237] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:03.209007 32522 catalog_manager.cc:5484] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130). New cstate: current_term: 1 leader_uuid: "07593a4e1c0242a1b52bb96338b63c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } }
W20240426 09:51:03.327008 1746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:03.327246 1746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:03.330252 1746 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240426 09:51:03.410704 464 raft_consensus.cc:1261] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:03.411151 594 raft_consensus.cc:1261] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:03.411720 1199 consensus_queue.cc:1035] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.412041 1199 consensus_queue.cc:1035] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.413821 463 raft_consensus.cc:1261] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:03.414093 32664 raft_consensus.cc:1261] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer 07593a4e1c0242a1b52bb96338b63c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:03.414387 1595 consensus_queue.cc:1035] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.414531 1315 consensus_queue.cc:1035] T 739229b67c4d49e7970c6cd9eb47690e P 07593a4e1c0242a1b52bb96338b63c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.419483 32665 raft_consensus.cc:1261] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:03.419893 326 raft_consensus.cc:1261] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:03.420289 1649 consensus_queue.cc:1035] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.422420 1649 consensus_queue.cc:1035] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 98.477 ms
time per row: 0.0492385 ms
I20240426 09:51:03.472411 32522 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53696:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240426 09:51:03.477851 444 tablet_service.cc:1467] Processing CreateTablet for tablet fed0f88283bb426f9a67435953f8a8b4 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ad7e6cdd130948aa9335ea31334ce6b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:03.477833 308 tablet_service.cc:1467] Processing CreateTablet for tablet fed0f88283bb426f9a67435953f8a8b4 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ad7e6cdd130948aa9335ea31334ce6b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:03.478006 574 tablet_service.cc:1467] Processing CreateTablet for tablet fed0f88283bb426f9a67435953f8a8b4 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ad7e6cdd130948aa9335ea31334ce6b8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:03.478150 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed0f88283bb426f9a67435953f8a8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:03.478233 574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed0f88283bb426f9a67435953f8a8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:03.478241 308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed0f88283bb426f9a67435953f8a8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:03.478843 307 tablet_service.cc:1467] Processing CreateTablet for tablet f3cc4098d5534bf6b5ba98d3988527ef (DEFAULT_TABLE table=rebalance_test_table_9 [id=ad7e6cdd130948aa9335ea31334ce6b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:03.479043 307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3cc4098d5534bf6b5ba98d3988527ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:03.479346 576 tablet_service.cc:1467] Processing CreateTablet for tablet f3cc4098d5534bf6b5ba98d3988527ef (DEFAULT_TABLE table=rebalance_test_table_9 [id=ad7e6cdd130948aa9335ea31334ce6b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:03.479568 576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3cc4098d5534bf6b5ba98d3988527ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:03.480620 301 tablet_service.cc:1467] Processing CreateTablet for tablet 8a57ddc4425d43d096b7da20ea1106b5 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ad7e6cdd130948aa9335ea31334ce6b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:03.481213 444 tablet_service.cc:1467] Processing CreateTablet for tablet f3cc4098d5534bf6b5ba98d3988527ef (DEFAULT_TABLE table=rebalance_test_table_9 [id=ad7e6cdd130948aa9335ea31334ce6b8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:03.481267 1811 tablet_bootstrap.cc:492] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:03.481420 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3cc4098d5534bf6b5ba98d3988527ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:03.481465 32643 tablet_service.cc:1467] Processing CreateTablet for tablet 8a57ddc4425d43d096b7da20ea1106b5 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ad7e6cdd130948aa9335ea31334ce6b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:03.481715 32643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a57ddc4425d43d096b7da20ea1106b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:03.481736 559 tablet_service.cc:1467] Processing CreateTablet for tablet 8a57ddc4425d43d096b7da20ea1106b5 (DEFAULT_TABLE table=rebalance_test_table_9 [id=ad7e6cdd130948aa9335ea31334ce6b8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240426 09:51:03.481917 559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a57ddc4425d43d096b7da20ea1106b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:03.482067 1811 tablet_bootstrap.cc:654] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:03.483040 1810 tablet_bootstrap.cc:492] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:03.483806 1814 tablet_bootstrap.cc:492] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2: Bootstrap starting.
I20240426 09:51:03.483862 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a57ddc4425d43d096b7da20ea1106b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 09:51:03.484297 1810 tablet_bootstrap.cc:654] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:03.484892 1814 tablet_bootstrap.cc:654] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:03.485561 1812 tablet_bootstrap.cc:492] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:03.486244 1814 tablet_bootstrap.cc:492] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2: No bootstrap required, opened a new log
I20240426 09:51:03.486294 1810 tablet_bootstrap.cc:492] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:03.486431 1812 tablet_bootstrap.cc:654] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:03.486419 1814 ts_tablet_manager.cc:1367] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:51:03.486428 1810 ts_tablet_manager.cc:1367] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:51:03.486984 1814 raft_consensus.cc:348] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.487272 1814 raft_consensus.cc:374] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:03.487323 1814 raft_consensus.cc:729] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Initialized, Role: FOLLOWER
I20240426 09:51:03.487450 1814 consensus_queue.cc:260] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.487617 1811 tablet_bootstrap.cc:492] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:03.487707 1811 ts_tablet_manager.cc:1367] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240426 09:51:03.487752 1814 ts_tablet_manager.cc:1398] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:03.487624 1810 raft_consensus.cc:348] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.487917 1810 raft_consensus.cc:374] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:03.487972 1810 raft_consensus.cc:729] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:03.488109 1811 raft_consensus.cc:348] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.488128 1810 consensus_queue.cc:260] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.488207 1811 raft_consensus.cc:374] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:03.488240 1811 raft_consensus.cc:729] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:03.488337 1811 consensus_queue.cc:260] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.488368 1810 ts_tablet_manager.cc:1398] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:03.488584 1811 ts_tablet_manager.cc:1398] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:03.488626 1810 tablet_bootstrap.cc:492] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b: Bootstrap starting.
I20240426 09:51:03.488734 1811 tablet_bootstrap.cc:492] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:03.489432 1810 tablet_bootstrap.cc:654] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:03.489559 1811 tablet_bootstrap.cc:654] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:03.490564 1812 tablet_bootstrap.cc:492] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:03.490657 1812 ts_tablet_manager.cc:1367] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240426 09:51:03.491078 1812 raft_consensus.cc:348] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.491230 1812 raft_consensus.cc:374] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:03.491300 1812 raft_consensus.cc:729] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:03.491446 1812 consensus_queue.cc:260] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.491591 1811 tablet_bootstrap.cc:492] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:03.491685 1811 ts_tablet_manager.cc:1367] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:51:03.492141 1811 raft_consensus.cc:348] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.492240 1811 raft_consensus.cc:374] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:03.492288 1811 raft_consensus.cc:729] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:03.492408 1811 consensus_queue.cc:260] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.492458 1812 ts_tablet_manager.cc:1398] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240426 09:51:03.492633 1812 tablet_bootstrap.cc:492] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:03.492673 1811 ts_tablet_manager.cc:1398] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240426 09:51:03.492828 1811 tablet_bootstrap.cc:492] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c: Bootstrap starting.
I20240426 09:51:03.493587 1811 tablet_bootstrap.cc:654] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:03.493708 1812 tablet_bootstrap.cc:654] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:03.494655 1811 tablet_bootstrap.cc:492] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c: No bootstrap required, opened a new log
I20240426 09:51:03.494803 1811 ts_tablet_manager.cc:1367] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240426 09:51:03.495267 1811 raft_consensus.cc:348] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.495442 1811 raft_consensus.cc:374] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:03.495522 1811 raft_consensus.cc:729] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Initialized, Role: FOLLOWER
I20240426 09:51:03.495731 1811 consensus_queue.cc:260] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.496189 1810 tablet_bootstrap.cc:492] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b: No bootstrap required, opened a new log
I20240426 09:51:03.496276 1810 ts_tablet_manager.cc:1367] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240426 09:51:03.496381 1811 ts_tablet_manager.cc:1398] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:03.496714 1810 raft_consensus.cc:348] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.497222 1812 tablet_bootstrap.cc:492] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:03.497277 1812 ts_tablet_manager.cc:1367] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240426 09:51:03.497388 1810 raft_consensus.cc:374] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:03.497479 1810 raft_consensus.cc:729] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Initialized, Role: FOLLOWER
I20240426 09:51:03.497615 1810 consensus_queue.cc:260] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.497912 1812 raft_consensus.cc:348] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.497980 1810 ts_tablet_manager.cc:1398] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:03.498024 1812 raft_consensus.cc:374] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:03.498056 1812 raft_consensus.cc:729] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:03.498179 1812 consensus_queue.cc:260] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.498629 1812 ts_tablet_manager.cc:1398] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240426 09:51:03.498808 1812 tablet_bootstrap.cc:492] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623: Bootstrap starting.
I20240426 09:51:03.499776 1812 tablet_bootstrap.cc:654] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623: Neither blocks nor log segments found. Creating new log.
I20240426 09:51:03.501703 1812 tablet_bootstrap.cc:492] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623: No bootstrap required, opened a new log
I20240426 09:51:03.501816 1812 ts_tablet_manager.cc:1367] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240426 09:51:03.502189 1812 raft_consensus.cc:348] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.502269 1812 raft_consensus.cc:374] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 09:51:03.502301 1812 raft_consensus.cc:729] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Initialized, Role: FOLLOWER
I20240426 09:51:03.502424 1812 consensus_queue.cc:260] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.502686 1812 ts_tablet_manager.cc:1398] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240426 09:51:03.518296 1725 raft_consensus.cc:482] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:03.518446 1725 raft_consensus.cc:504] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.518914 1725 leader_election.cc:288] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:03.519282 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a57ddc4425d43d096b7da20ea1106b5" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:03.519286 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a57ddc4425d43d096b7da20ea1106b5" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623" is_pre_election: true
I20240426 09:51:03.519400 594 raft_consensus.cc:2454] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:03.519423 327 raft_consensus.cc:2454] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 0.
I20240426 09:51:03.519714 32598 leader_election.cc:302] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:03.519918 1725 raft_consensus.cc:2781] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:03.519975 1725 raft_consensus.cc:482] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:03.520009 1725 raft_consensus.cc:3037] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.520759 1725 raft_consensus.cc:504] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.521100 1725 leader_election.cc:288] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:03.521432 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a57ddc4425d43d096b7da20ea1106b5" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:03.521432 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a57ddc4425d43d096b7da20ea1106b5" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:03.521549 594 raft_consensus.cc:3037] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.521549 327 raft_consensus.cc:3037] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.522488 327 raft_consensus.cc:2454] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:03.522488 594 raft_consensus.cc:2454] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 1.
I20240426 09:51:03.522825 32598 leader_election.cc:302] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:03.523026 1725 raft_consensus.cc:2781] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:03.523104 1725 raft_consensus.cc:686] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:03.523263 1725 consensus_queue.cc:237] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.524930 32522 catalog_manager.cc:5484] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:03.528107 1806 raft_consensus.cc:482] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:03.528189 1806 raft_consensus.cc:504] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.528534 1806 leader_election.cc:288] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:03.528874 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed0f88283bb426f9a67435953f8a8b4" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:03.528997 327 raft_consensus.cc:2454] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:03.528962 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed0f88283bb426f9a67435953f8a8b4" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:03.529106 464 raft_consensus.cc:2454] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:03.529521 532 leader_election.cc:302] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:03.529685 1806 raft_consensus.cc:2781] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:03.529742 1806 raft_consensus.cc:482] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:03.529779 1806 raft_consensus.cc:3037] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.530547 1806 raft_consensus.cc:504] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.530900 1806 leader_election.cc:288] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:03.531167 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed0f88283bb426f9a67435953f8a8b4" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:03.531280 327 raft_consensus.cc:3037] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.531257 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed0f88283bb426f9a67435953f8a8b4" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:03.531359 464 raft_consensus.cc:3037] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.532176 327 raft_consensus.cc:2454] T fed0f88283bb426f9a67435953f8a8b4 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:03.532261 464 raft_consensus.cc:2454] T fed0f88283bb426f9a67435953f8a8b4 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:03.532485 532 leader_election.cc:302] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:03.532653 1806 raft_consensus.cc:2781] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:03.532785 1806 raft_consensus.cc:686] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:03.532946 1806 consensus_queue.cc:237] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.534545 32522 catalog_manager.cc:5484] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:03.759073 1649 raft_consensus.cc:482] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 09:51:03.759187 1649 raft_consensus.cc:504] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.759505 1649 leader_election.cc:288] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:03.759900 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cc4098d5534bf6b5ba98d3988527ef" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c" is_pre_election: true
I20240426 09:51:03.759922 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cc4098d5534bf6b5ba98d3988527ef" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" is_pre_election: true
I20240426 09:51:03.760044 326 raft_consensus.cc:2454] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:03.760058 464 raft_consensus.cc:2454] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 0.
I20240426 09:51:03.760318 532 leader_election.cc:302] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:03.760499 1649 raft_consensus.cc:2781] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 09:51:03.760560 1649 raft_consensus.cc:482] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 09:51:03.760586 1649 raft_consensus.cc:3037] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.761415 1649 raft_consensus.cc:504] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.761731 1649 leader_election.cc:288] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:03.762065 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cc4098d5534bf6b5ba98d3988527ef" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:03.762082 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cc4098d5534bf6b5ba98d3988527ef" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:03.762189 326 raft_consensus.cc:3037] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.762189 464 raft_consensus.cc:3037] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b [term 0 FOLLOWER]: Advancing to term 1
I20240426 09:51:03.763134 464 raft_consensus.cc:2454] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:03.763134 326 raft_consensus.cc:2454] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 1.
I20240426 09:51:03.763432 532 leader_election.cc:302] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:03.763608 1649 raft_consensus.cc:2781] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 09:51:03.763705 1649 raft_consensus.cc:686] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:03.763842 1649 consensus_queue.cc:237] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:03.765569 32522 catalog_manager.cc:5484] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 1 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
W20240426 09:51:03.889621 1826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:03.890044 1826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:03.893563 1826 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240426 09:51:03.965590 1649 consensus_queue.cc:1035] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.969022 1806 consensus_queue.cc:1035] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.972937 1725 consensus_queue.cc:1035] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.982316 326 raft_consensus.cc:1261] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:03.982713 1725 consensus_queue.cc:1035] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.985970 464 raft_consensus.cc:1261] T f3cc4098d5534bf6b5ba98d3988527ef P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:03.986003 327 raft_consensus.cc:1261] T f3cc4098d5534bf6b5ba98d3988527ef P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 09:51:03.986377 1806 consensus_queue.cc:1035] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 09:51:03.986500 1649 consensus_queue.cc:1035] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 125.114 ms
time per row: 0.0625572 ms
I20240426 09:51:04.059067 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.133:42497
--local_ip_for_outbound_sockets=127.0.99.133
--tserver_master_addrs=127.0.99.190:37003
--webserver_port=36651
--webserver_interface=127.0.99.133
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240426 09:51:04.177263 1887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:04.177551 1887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:51:04.177613 1887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:04.181828 1887 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:51:04.181917 1887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:51:04.182014 1887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.133
I20240426 09:51:04.187009 1887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.133:42497
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.0.99.133
--webserver_port=36651
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.1887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.133
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:51:04.188088 1887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:51:04.189406 1887 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:51:04.198061 1895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:51:04.198935 1892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:51:04.199353 1893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:51:04.200292 1887 server_base.cc:1034] running on GCE node
I20240426 09:51:04.200637 1887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:51:04.201138 1887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:51:04.202303 1887 hybrid_clock.cc:648] HybridClock initialized: now 1714125064202281 us; error 34 us; skew 500 ppm
I20240426 09:51:04.208240 1887 webserver.cc:460] Webserver started at http://127.0.99.133:36651/ using document root <none> and password file <none>
I20240426 09:51:04.208770 1887 fs_manager.cc:359] Metadata directory not provided
I20240426 09:51:04.208842 1887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:51:04.212072 1887 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20240426 09:51:04.213869 1901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.214910 1887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240426 09:51:04.215025 1887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "c252379311f0425ca38566356d6b41c2"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:51:04.215451 1887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:51:04.239764 1887 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:51:04.240384 1887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:51:04.240548 1887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:51:04.241101 1887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:51:04.241935 1887 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:51:04.241978 1887 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.242033 1887 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:51:04.242063 1887 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.250639 1887 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.133:42497
I20240426 09:51:04.250672 2014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.133:42497 every 8 connection(s)
I20240426 09:51:04.251693 1887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240426 09:51:04.255103 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 1887
I20240426 09:51:04.255538 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.134:35801
--local_ip_for_outbound_sockets=127.0.99.134
--tserver_master_addrs=127.0.99.190:37003
--webserver_port=37693
--webserver_interface=127.0.99.134
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:51:04.260705 2015 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:51:04.260929 2015 heartbeater.cc:464] Registering TS with master...
I20240426 09:51:04.261348 2015 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:51:04.262347 32522 ts_manager.cc:194] Re-registered known tserver with Master: c252379311f0425ca38566356d6b41c2 (127.0.99.133:42497)
I20240426 09:51:04.262949 32522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.133:41833
W20240426 09:51:04.362895 2018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:04.363176 2018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:51:04.363252 2018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:04.366545 2018 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:51:04.366621 2018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:51:04.366709 2018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.134
I20240426 09:51:04.370404 2018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.134:35801
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.0.99.134
--webserver_port=37693
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.2018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.134
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:51:04.371322 2018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:51:04.372429 2018 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:51:04.381012 2026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:51:04.381457 2023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:51:04.382357 2024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:51:04.383327 2018 server_base.cc:1034] running on GCE node
I20240426 09:51:04.383764 2018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:51:04.384407 2018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:51:04.385588 2018 hybrid_clock.cc:648] HybridClock initialized: now 1714125064385561 us; error 46 us; skew 500 ppm
I20240426 09:51:04.387881 2018 webserver.cc:460] Webserver started at http://127.0.99.134:37693/ using document root <none> and password file <none>
I20240426 09:51:04.388412 2018 fs_manager.cc:359] Metadata directory not provided
I20240426 09:51:04.388489 2018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:51:04.391906 2018 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240426 09:51:04.393894 2032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.395293 2018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240426 09:51:04.395403 2018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "b22f98253a344ccd969920c43529f24e"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:51:04.395898 2018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:51:04.409817 2018 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:51:04.410408 2018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:51:04.410562 2018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:51:04.411166 2018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:51:04.412086 2018 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:51:04.412149 2018 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.412206 2018 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:51:04.412228 2018 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.420908 2018 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.134:35801
I20240426 09:51:04.420958 2145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.134:35801 every 8 connection(s)
I20240426 09:51:04.421936 2018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240426 09:51:04.422487 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2018
I20240426 09:51:04.422937 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.135:46491
--local_ip_for_outbound_sockets=127.0.99.135
--tserver_master_addrs=127.0.99.190:37003
--webserver_port=32921
--webserver_interface=127.0.99.135
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:51:04.431196 2146 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:51:04.431414 2146 heartbeater.cc:464] Registering TS with master...
I20240426 09:51:04.431854 2146 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:51:04.432816 32522 ts_manager.cc:194] Re-registered known tserver with Master: b22f98253a344ccd969920c43529f24e (127.0.99.134:35801)
I20240426 09:51:04.433630 32522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.134:39547
W20240426 09:51:04.534685 2149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:04.534983 2149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:51:04.535033 2149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:04.537964 2149 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:51:04.538020 2149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:51:04.538082 2149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.135
I20240426 09:51:04.541709 2149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.135:46491
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.0.99.135
--webserver_port=32921
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.2149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.135
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:51:04.542557 2149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:51:04.543668 2149 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:51:04.552894 2155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:51:04.553700 2158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:51:04.554177 2156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:51:04.555122 2149 server_base.cc:1034] running on GCE node
I20240426 09:51:04.555521 2149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:51:04.556094 2149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:51:04.557246 2149 hybrid_clock.cc:648] HybridClock initialized: now 1714125064557233 us; error 27 us; skew 500 ppm
I20240426 09:51:04.559255 2149 webserver.cc:460] Webserver started at http://127.0.99.135:32921/ using document root <none> and password file <none>
I20240426 09:51:04.559809 2149 fs_manager.cc:359] Metadata directory not provided
I20240426 09:51:04.559885 2149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:51:04.563462 2149 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240426 09:51:04.565196 2164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.566152 2149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240426 09:51:04.566303 2149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "777b53878fab4bc885635b63dd63ae2e"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:51:04.566802 2149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:51:04.596892 2149 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:51:04.597486 2149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:51:04.597656 2149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:51:04.598176 2149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:51:04.599045 2149 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:51:04.599090 2149 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.599134 2149 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:51:04.599164 2149 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.607923 2149 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.135:46491
I20240426 09:51:04.607949 2278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.135:46491 every 8 connection(s)
I20240426 09:51:04.608883 2149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240426 09:51:04.609594 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2149
I20240426 09:51:04.610031 398 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
/tmp/dist-test-taskrLZDS8/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.99.136:37421
--local_ip_for_outbound_sockets=127.0.99.136
--tserver_master_addrs=127.0.99.190:37003
--webserver_port=34675
--webserver_interface=127.0.99.136
--builtin_ntp_servers=127.0.99.148:43235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240426 09:51:04.618474 2279 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:51:04.618716 2279 heartbeater.cc:464] Registering TS with master...
I20240426 09:51:04.619643 2279 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:51:04.621057 32522 ts_manager.cc:194] Re-registered known tserver with Master: 777b53878fab4bc885635b63dd63ae2e (127.0.99.135:46491)
I20240426 09:51:04.621567 32522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.135:56347
W20240426 09:51:04.724961 2281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240426 09:51:04.725167 2281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240426 09:51:04.725241 2281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240426 09:51:04.728147 2281 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240426 09:51:04.728209 2281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240426 09:51:04.728288 2281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.99.136
I20240426 09:51:04.732376 2281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.99.148:43235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.99.136:37421
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.0.99.136
--webserver_port=34675
--tserver_master_addrs=127.0.99.190:37003
--never_fsync=true
--heap_profile_path=/tmp/kudu.2281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.99.136
--log_dir=/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 058c4bd848277b49fc34b19c81c39a215d7f637f
build type DEBUG
built by None at 26 Apr 2024 09:35:15 UTC on efd1337a2aa3
build id 257
I20240426 09:51:04.733289 2281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 09:51:04.734416 2281 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 09:51:04.743238 2290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:51:04.743705 2287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 09:51:04.744210 2288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 09:51:04.745134 2281 server_base.cc:1034] running on GCE node
I20240426 09:51:04.745499 2281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240426 09:51:04.746094 2281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240426 09:51:04.747248 2281 hybrid_clock.cc:648] HybridClock initialized: now 1714125064747239 us; error 31 us; skew 500 ppm
I20240426 09:51:04.749058 2281 webserver.cc:460] Webserver started at http://127.0.99.136:34675/ using document root <none> and password file <none>
I20240426 09:51:04.749588 2281 fs_manager.cc:359] Metadata directory not provided
I20240426 09:51:04.749673 2281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 09:51:04.753001 2281 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20240426 09:51:04.754943 2296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.755970 2281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240426 09:51:04.756069 2281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e87b815a8bac4e03ab414f74df93f02a"
format_stamp: "Formatted at 2024-04-26 09:50:54 on dist-test-slave-1zqj"
I20240426 09:51:04.756467 2281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 09:51:04.777930 2281 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240426 09:51:04.778611 2281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 09:51:04.778805 2281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 09:51:04.779418 2281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 09:51:04.780404 2281 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 09:51:04.780452 2281 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.780493 2281 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 09:51:04.780526 2281 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240426 09:51:04.789465 2281 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.136:37421
I20240426 09:51:04.789515 2409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.136:37421 every 8 connection(s)
I20240426 09:51:04.790468 2281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskrLZDS8/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714124921875691-398-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240426 09:51:04.796291 398 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskrLZDS8/build/debug/bin/kudu as pid 2281
I20240426 09:51:04.796525 398 test_util.cc:268] Using random seed: -678004374
I20240426 09:51:04.799803 2410 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37003
I20240426 09:51:04.800031 2410 heartbeater.cc:464] Registering TS with master...
I20240426 09:51:04.800482 2410 heartbeater.cc:511] Master 127.0.99.190:37003 requested a full tablet report, sending...
I20240426 09:51:04.801546 32522 ts_manager.cc:194] Re-registered known tserver with Master: e87b815a8bac4e03ab414f74df93f02a (127.0.99.136:37421)
I20240426 09:51:04.802244 32522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.99.136:50551
I20240426 09:51:04.807890 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240426 09:51:04.808135 398 test_util.cc:268] Using random seed: -677992766
I20240426 09:51:04.816534 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f3cc4098d5534bf6b5ba98d3988527ef"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.816735 594 raft_consensus.cc:465] T f3cc4098d5534bf6b5ba98d3988527ef P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.817301 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b8ebf7b59324fcc868332f67b6fef1d"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.817409 594 raft_consensus.cc:465] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.818048 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.818153 594 raft_consensus.cc:482] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.818212 594 raft_consensus.cc:3037] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.819409 594 raft_consensus.cc:504] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.819804 594 leader_election.cc:288] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:04.820179 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.820302 32664 raft_consensus.cc:3032] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.820341 32664 raft_consensus.cc:729] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.820464 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.820500 32664 consensus_queue.cc:260] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.820552 594 raft_consensus.cc:482] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.820621 594 raft_consensus.cc:3037] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.820780 32664 raft_consensus.cc:3037] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.821491 594 raft_consensus.cc:504] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.821575 32664 raft_consensus.cc:2454] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.821835 594 leader_election.cc:288] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:04.821833 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.821964 463 raft_consensus.cc:3037] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.822515 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dff34820cccc4ba2b7213724d8969ddb"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.822635 594 raft_consensus.cc:465] T dff34820cccc4ba2b7213724d8969ddb P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.823011 463 raft_consensus.cc:2454] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.823289 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "713fa4d62643470285727c0d0092676f"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.823400 594 raft_consensus.cc:482] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.823468 594 raft_consensus.cc:3037] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.824081 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.824124 532 leader_election.cc:302] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.824189 464 raft_consensus.cc:3032] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.824234 464 raft_consensus.cc:729] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4076eb0699724ee78c2f2f0d6354cf9b, State: Running, Role: LEADER
I20240426 09:51:04.824257 594 raft_consensus.cc:504] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.824374 464 consensus_queue.cc:260] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.824555 594 leader_election.cc:288] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:04.824584 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240426 09:51:04.824618 464 raft_consensus.cc:3037] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.824715 398 test_util.cc:268] Using random seed: -677976182
I20240426 09:51:04.825421 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8a57ddc4425d43d096b7da20ea1106b5"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.825500 464 raft_consensus.cc:2454] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.825529 594 raft_consensus.cc:482] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.825577 594 raft_consensus.cc:3037] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.825661 2153 raft_consensus.cc:2781] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.825737 2153 raft_consensus.cc:686] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.825851 2153 consensus_queue.cc:237] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.826367 594 raft_consensus.cc:504] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.827402 594 leader_election.cc:288] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:04.827467 532 leader_election.cc:302] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.827960 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a57ddc4425d43d096b7da20ea1106b5" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.828018 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f05415a6ed614f95be718921868fff20"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.828084 327 raft_consensus.cc:3037] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.828110 594 raft_consensus.cc:465] T f05415a6ed614f95be718921868fff20 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.828106 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713fa4d62643470285727c0d0092676f" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.828200 32665 raft_consensus.cc:3032] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.828243 32665 raft_consensus.cc:729] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.828370 32665 consensus_queue.cc:260] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.828495 32665 raft_consensus.cc:3037] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.829073 327 raft_consensus.cc:2454] T 8a57ddc4425d43d096b7da20ea1106b5 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.829320 32665 raft_consensus.cc:2454] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.829349 32663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a57ddc4425d43d096b7da20ea1106b5" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.829457 32663 raft_consensus.cc:3032] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.829495 32663 raft_consensus.cc:729] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.829512 1867 raft_consensus.cc:2781] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.829602 1867 raft_consensus.cc:686] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.829665 32663 consensus_queue.cc:260] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.829718 1867 consensus_queue.cc:237] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.829852 32663 raft_consensus.cc:3037] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.830332 532 leader_election.cc:302] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.830494 532 leader_election.cc:302] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.830628 1867 raft_consensus.cc:2781] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.830809 1867 raft_consensus.cc:686] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.830825 32663 raft_consensus.cc:2454] T 8a57ddc4425d43d096b7da20ea1106b5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.830948 1867 consensus_queue.cc:237] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.831418 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.831519 32664 raft_consensus.cc:3037] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.830628 2153 raft_consensus.cc:2781] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.832027 2153 raft_consensus.cc:686] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.832129 2153 consensus_queue.cc:237] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.832367 32664 raft_consensus.cc:2454] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.832746 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713fa4d62643470285727c0d0092676f" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.832886 463 raft_consensus.cc:3037] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.833099 32522 catalog_manager.cc:5484] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:04.833297 32522 catalog_manager.cc:5484] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:04.833753 463 raft_consensus.cc:2454] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.834159 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.834255 594 raft_consensus.cc:482] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.834300 594 raft_consensus.cc:3037] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.835142 594 raft_consensus.cc:504] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.835412 594 leader_election.cc:288] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:04.835873 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.835912 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.835963 32664 raft_consensus.cc:3032] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.835999 32664 raft_consensus.cc:729] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.836012 463 raft_consensus.cc:3037] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.836109 32664 consensus_queue.cc:260] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.836256 32664 raft_consensus.cc:3037] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.836488 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "679d7a8559df4e6fbae5e44dc3b2e227"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.836587 594 raft_consensus.cc:482] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.836658 594 raft_consensus.cc:3037] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.836843 463 raft_consensus.cc:2454] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.837095 532 leader_election.cc:302] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.837121 32664 raft_consensus.cc:2454] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.837440 2153 raft_consensus.cc:2781] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.837510 2153 raft_consensus.cc:686] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.837507 594 raft_consensus.cc:504] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.837658 2153 consensus_queue.cc:237] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.837844 594 leader_election.cc:288] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:04.838169 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679d7a8559df4e6fbae5e44dc3b2e227" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.838176 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679d7a8559df4e6fbae5e44dc3b2e227" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.838254 32664 raft_consensus.cc:3032] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.838289 32664 raft_consensus.cc:729] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.838294 327 raft_consensus.cc:3037] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.838438 32664 consensus_queue.cc:260] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.838604 32664 raft_consensus.cc:3037] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.838646 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.838730 594 raft_consensus.cc:482] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.838797 594 raft_consensus.cc:3037] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.839287 327 raft_consensus.cc:2454] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.839547 32664 raft_consensus.cc:2454] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.839584 532 leader_election.cc:302] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.839545 594 raft_consensus.cc:504] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.839839 594 leader_election.cc:288] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:04.840129 2153 raft_consensus.cc:2781] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.840193 2153 raft_consensus.cc:686] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.840296 2153 consensus_queue.cc:237] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.840391 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.840478 595 raft_consensus.cc:482] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.840533 595 raft_consensus.cc:3037] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.840767 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.840864 463 raft_consensus.cc:3037] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.841377 595 raft_consensus.cc:504] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.841672 595 leader_election.cc:288] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:04.841711 463 raft_consensus.cc:2454] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.842120 532 leader_election.cc:302] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.842160 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.842252 327 raft_consensus.cc:3037] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.842272 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b2ecfdfe7ac44636b5d4b620c05ed1c9"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.842362 595 raft_consensus.cc:465] T b2ecfdfe7ac44636b5d4b620c05ed1c9 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.841996 32664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.842687 32664 raft_consensus.cc:3032] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.842731 32664 raft_consensus.cc:729] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.842913 32664 consensus_queue.cc:260] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.843060 32664 raft_consensus.cc:3037] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.843065 327 raft_consensus.cc:2454] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.843307 532 leader_election.cc:302] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.843466 1867 raft_consensus.cc:2781] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.843524 1867 raft_consensus.cc:686] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.843628 1867 consensus_queue.cc:237] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.843648 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9d752855971a4015b1ebc62993ea9b22"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.843719 32663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.843744 595 raft_consensus.cc:465] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.843798 32663 raft_consensus.cc:3032] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.843834 32663 raft_consensus.cc:729] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.843945 32663 consensus_queue.cc:260] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.844091 32663 raft_consensus.cc:3037] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.844210 1867 raft_consensus.cc:2781] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.844280 1867 raft_consensus.cc:686] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.844375 1867 consensus_queue.cc:237] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.844571 32664 raft_consensus.cc:2454] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.845006 32663 raft_consensus.cc:2454] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.845970 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.846081 595 raft_consensus.cc:482] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.846138 595 raft_consensus.cc:3037] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.846868 595 raft_consensus.cc:504] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.847151 595 leader_election.cc:288] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:04.847708 32663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.847800 32663 raft_consensus.cc:3032] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.847831 32663 raft_consensus.cc:729] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.847934 32663 consensus_queue.cc:260] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.847996 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4584b80c6d7e4feca28d7b321bca613f"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.848055 32663 raft_consensus.cc:3037] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.848093 595 raft_consensus.cc:482] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.848152 595 raft_consensus.cc:3037] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.848810 32663 raft_consensus.cc:2454] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.848986 595 raft_consensus.cc:504] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.849295 595 leader_election.cc:288] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
W20240426 09:51:04.849633 32624 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33868: Illegal state: replica 342778fb25b446c68cdcd20f98bcd8b2 is not leader of this config: current role FOLLOWER
I20240426 09:51:04.849679 532 leader_election.cc:302] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.849788 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4584b80c6d7e4feca28d7b321bca613f" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.849910 463 raft_consensus.cc:3037] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.850220 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4584b80c6d7e4feca28d7b321bca613f" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.850347 326 raft_consensus.cc:3032] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.850364 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.850407 326 raft_consensus.cc:729] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:04.850440 327 raft_consensus.cc:3037] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.850592 326 consensus_queue.cc:260] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.850845 1867 raft_consensus.cc:2781] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.850919 326 raft_consensus.cc:3037] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.850945 1867 raft_consensus.cc:686] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.851063 1867 consensus_queue.cc:237] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.851348 327 raft_consensus.cc:2454] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.851707 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d6ad2b31cf884e62898c75d2426d8400"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.851750 463 raft_consensus.cc:2454] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.851827 595 raft_consensus.cc:482] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.851866 326 raft_consensus.cc:2454] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.851893 595 raft_consensus.cc:3037] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.852658 532 leader_election.cc:302] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.852847 2153 raft_consensus.cc:2781] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.852944 2153 raft_consensus.cc:686] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.853058 2153 consensus_queue.cc:237] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.853775 595 raft_consensus.cc:504] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.854089 595 leader_election.cc:288] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:04.854312 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad2b31cf884e62898c75d2426d8400" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.854418 463 raft_consensus.cc:3037] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.854496 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6ad2b31cf884e62898c75d2426d8400" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.854729 327 raft_consensus.cc:3032] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.854830 327 raft_consensus.cc:729] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:04.854812 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "be27b88cb26b48b499de234f7c9de3e2"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.854902 595 raft_consensus.cc:465] T be27b88cb26b48b499de234f7c9de3e2 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.855389 463 raft_consensus.cc:2454] T d6ad2b31cf884e62898c75d2426d8400 P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.855696 532 leader_election.cc:302] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4076eb0699724ee78c2f2f0d6354cf9b, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.855885 2444 raft_consensus.cc:2781] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.855424 327 consensus_queue.cc:260] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.855978 2444 raft_consensus.cc:686] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.856052 327 raft_consensus.cc:3037] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.856101 2444 consensus_queue.cc:237] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.856915 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fed0f88283bb426f9a67435953f8a8b4"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.857084 595 raft_consensus.cc:465] T fed0f88283bb426f9a67435953f8a8b4 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.857620 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d12dc2eddc8c402fb0f12c97410d9af2"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.857709 595 raft_consensus.cc:465] T d12dc2eddc8c402fb0f12c97410d9af2 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.858170 594 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c8353b8e1494e33a069eb6921b81ba5"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.858297 594 raft_consensus.cc:465] T 3c8353b8e1494e33a069eb6921b81ba5 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.858929 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f34e4326a7cd44669b9a58324d883911"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.859014 595 raft_consensus.cc:482] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.859059 595 raft_consensus.cc:3037] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.859949 327 raft_consensus.cc:2454] T d6ad2b31cf884e62898c75d2426d8400 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.860167 595 raft_consensus.cc:504] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.860406 32663 raft_consensus.cc:1261] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.860489 595 leader_election.cc:288] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:04.860891 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34e4326a7cd44669b9a58324d883911" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.860994 326 raft_consensus.cc:3037] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.861100 595 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "786a6056e8164484b976dc64b1aed9ae"
dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
from {username='slave'} at 127.0.0.1:58562
I20240426 09:51:04.861129 327 raft_consensus.cc:1261] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.861200 595 raft_consensus.cc:482] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.861253 595 raft_consensus.cc:3037] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.861843 326 raft_consensus.cc:2454] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.862013 595 raft_consensus.cc:504] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.862236 32663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34e4326a7cd44669b9a58324d883911" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.862308 595 leader_election.cc:288] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129:36999)
I20240426 09:51:04.862365 32663 raft_consensus.cc:3032] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.862409 32663 raft_consensus.cc:729] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.862475 532 leader_election.cc:302] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.862524 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240426 09:51:04.862555 32663 consensus_queue.cc:260] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.862674 398 test_util.cc:268] Using random seed: -677938223
I20240426 09:51:04.862682 2153 consensus_queue.cc:1035] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.862730 32663 raft_consensus.cc:3037] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.862977 2153 consensus_queue.cc:1035] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.863178 2153 raft_consensus.cc:2781] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.863231 2153 raft_consensus.cc:686] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.863322 2153 consensus_queue.cc:237] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.863729 32663 raft_consensus.cc:2454] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.864316 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a6056e8164484b976dc64b1aed9ae" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.864410 326 raft_consensus.cc:3032] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.864459 326 raft_consensus.cc:729] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:04.864567 32663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "786a6056e8164484b976dc64b1aed9ae" candidate_uuid: "f9655c9ff2e14edfbcd798d88bca0623" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
I20240426 09:51:04.864589 326 consensus_queue.cc:260] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.864658 32663 raft_consensus.cc:3037] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.864748 326 raft_consensus.cc:3037] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.865440 32663 raft_consensus.cc:2454] T 786a6056e8164484b976dc64b1aed9ae P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
I20240426 09:51:04.865525 326 raft_consensus.cc:2454] T 786a6056e8164484b976dc64b1aed9ae P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9655c9ff2e14edfbcd798d88bca0623 in term 2.
W20240426 09:51:04.865535 32624 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33854: Illegal state: replica 342778fb25b446c68cdcd20f98bcd8b2 is not leader of this config: current role FOLLOWER
I20240426 09:51:04.865996 532 leader_election.cc:302] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.866297 1867 raft_consensus.cc:2781] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.866380 1867 raft_consensus.cc:686] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Leader. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.866479 1867 consensus_queue.cc:237] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.868677 32514 catalog_manager.cc:5484] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c reported cstate change: term changed from 1 to 2, leader changed from 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } }
I20240426 09:51:04.874178 32664 raft_consensus.cc:1261] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.874249 463 raft_consensus.cc:1261] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.874583 1867 consensus_queue.cc:1035] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.874914 2444 consensus_queue.cc:1035] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.878167 32518 catalog_manager.cc:5484] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b reported cstate change: term changed from 1 to 2, leader changed from 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
W20240426 09:51:04.887094 32624 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33870: Illegal state: replica 342778fb25b446c68cdcd20f98bcd8b2 is not leader of this config: current role FOLLOWER
I20240426 09:51:04.888774 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240426 09:51:04.888940 398 test_util.cc:268] Using random seed: -677911957
I20240426 09:51:04.895278 32517 catalog_manager.cc:5484] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:04.895529 32517 catalog_manager.cc:5484] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
I20240426 09:51:04.895713 32517 catalog_manager.cc:5484] T d6ad2b31cf884e62898c75d2426d8400 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } }
I20240426 09:51:04.895941 32517 catalog_manager.cc:5484] T 8a57ddc4425d43d096b7da20ea1106b5 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:04.896116 32517 catalog_manager.cc:5484] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:04.896320 32517 catalog_manager.cc:5484] T 786a6056e8164484b976dc64b1aed9ae P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:04.896478 32517 catalog_manager.cc:5484] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } }
I20240426 09:51:04.896648 32517 catalog_manager.cc:5484] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
I20240426 09:51:04.896860 32517 catalog_manager.cc:5484] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 reported cstate change: term changed from 1 to 2, leader changed from 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129) to f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132). New cstate: current_term: 2 leader_uuid: "f9655c9ff2e14edfbcd798d88bca0623" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
W20240426 09:51:04.900884 32624 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33854: Illegal state: replica 342778fb25b446c68cdcd20f98bcd8b2 is not leader of this config: current role FOLLOWER
I20240426 09:51:04.904837 2441 mvcc.cc:205] Tried to move back new op lower bound from 7021056265721462784 to 7021056265548587008. Current Snapshot: MvccSnapshot[applied={T|T < 7021056265721462784 or (T in {7021056265721462784})}]
I20240426 09:51:04.915760 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.915935 32663 raft_consensus.cc:482] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.916005 32663 raft_consensus.cc:3037] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.917112 32663 raft_consensus.cc:504] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.917461 32663 leader_election.cc:288] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:04.918108 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.918224 463 raft_consensus.cc:3037] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.919282 463 raft_consensus.cc:2454] T 18d93e5032a04e069ff7a924078a9989 P 4076eb0699724ee78c2f2f0d6354cf9b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
I20240426 09:51:04.919513 32601 leader_election.cc:302] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:04.919674 1725 raft_consensus.cc:2781] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Leader election won for term 3
I20240426 09:51:04.919759 1725 raft_consensus.cc:686] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.919924 1725 consensus_queue.cc:237] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.920588 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d93e5032a04e069ff7a924078a9989" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.920697 595 raft_consensus.cc:3032] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Stepping down as leader of term 2
I20240426 09:51:04.920737 595 raft_consensus.cc:729] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.920914 595 consensus_queue.cc:260] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.921149 595 raft_consensus.cc:3037] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.921681 32522 catalog_manager.cc:5484] T 18d93e5032a04e069ff7a924078a9989 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 2 to 3, leader changed from f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132) to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 3 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:04.922103 595 raft_consensus.cc:2396] T 18d93e5032a04e069ff7a924078a9989 P f9655c9ff2e14edfbcd798d88bca0623 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 342778fb25b446c68cdcd20f98bcd8b2 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240426 09:51:04.922351 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b8ebf7b59324fcc868332f67b6fef1d"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.922449 32663 raft_consensus.cc:482] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.922497 32663 raft_consensus.cc:3037] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.923343 32663 raft_consensus.cc:504] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.923691 32663 leader_election.cc:288] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 2 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:04.924268 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ebf7b59324fcc868332f67b6fef1d" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.924324 32664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.924391 328 raft_consensus.cc:3037] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.924407 32664 raft_consensus.cc:482] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.924479 32664 raft_consensus.cc:3037] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.925300 328 raft_consensus.cc:2454] T 0b8ebf7b59324fcc868332f67b6fef1d P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 2.
I20240426 09:51:04.925300 32664 raft_consensus.cc:504] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.925578 32601 leader_election.cc:302] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:51:04.925841 2432 raft_consensus.cc:2781] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.925930 2432 raft_consensus.cc:686] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.926041 2432 consensus_queue.cc:237] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.927665 594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.927783 594 raft_consensus.cc:3032] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Stepping down as leader of term 2
I20240426 09:51:04.927834 594 raft_consensus.cc:729] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.927954 594 consensus_queue.cc:260] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.928078 594 raft_consensus.cc:3037] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.929003 594 raft_consensus.cc:2396] T c4fbe777c8f94d80b3ae5df7bef3b724 P f9655c9ff2e14edfbcd798d88bca0623 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 342778fb25b446c68cdcd20f98bcd8b2 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20240426 09:51:04.929386 32756 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:58562: Illegal state: replica 07593a4e1c0242a1b52bb96338b63c8c is not leader of this config: current role FOLLOWER
W20240426 09:51:04.929572 425 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:44760: Illegal state: replica 4076eb0699724ee78c2f2f0d6354cf9b is not leader of this config: current role FOLLOWER
I20240426 09:51:04.930264 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
I20240426 09:51:04.930430 398 test_util.cc:268] Using random seed: -677870468
I20240426 09:51:04.931506 32522 catalog_manager.cc:5484] T 0b8ebf7b59324fcc868332f67b6fef1d P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 1 to 2, leader changed from f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132) to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:04.932224 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4fbe777c8f94d80b3ae5df7bef3b724" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.932333 464 raft_consensus.cc:3037] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.933286 464 raft_consensus.cc:2454] T c4fbe777c8f94d80b3ae5df7bef3b724 P 4076eb0699724ee78c2f2f0d6354cf9b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
I20240426 09:51:04.925595 32664 leader_election.cc:288] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:04.933763 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b8ebf7b59324fcc868332f67b6fef1d" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.933861 595 raft_consensus.cc:3032] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.933905 595 raft_consensus.cc:729] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.934038 595 consensus_queue.cc:260] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.934250 595 raft_consensus.cc:3037] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.934311 32664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.934412 32664 raft_consensus.cc:482] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.934461 32664 raft_consensus.cc:3037] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.934785 328 raft_consensus.cc:1261] T 4584b80c6d7e4feca28d7b321bca613f P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.935166 1867 consensus_queue.cc:1035] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.935252 595 raft_consensus.cc:2454] T 0b8ebf7b59324fcc868332f67b6fef1d P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 2.
I20240426 09:51:04.935400 32664 raft_consensus.cc:504] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.935765 32664 leader_election.cc:288] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:04.937004 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.937105 595 raft_consensus.cc:3032] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Stepping down as leader of term 2
I20240426 09:51:04.937148 595 raft_consensus.cc:729] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.937263 595 consensus_queue.cc:260] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.937376 595 raft_consensus.cc:3037] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.938266 595 raft_consensus.cc:2396] T 2eae2c486afa483ab3ed754cc720ed8f P f9655c9ff2e14edfbcd798d88bca0623 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 342778fb25b446c68cdcd20f98bcd8b2 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240426 09:51:04.938447 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9d752855971a4015b1ebc62993ea9b22"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.938544 32663 raft_consensus.cc:482] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.938594 32663 raft_consensus.cc:3037] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.938866 32664 raft_consensus.cc:1261] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.939461 32663 raft_consensus.cc:504] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.939831 32663 leader_election.cc:288] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 2 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:04.940704 1867 consensus_queue.cc:1035] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.941031 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d752855971a4015b1ebc62993ea9b22" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.941138 328 raft_consensus.cc:3037] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.941331 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d752855971a4015b1ebc62993ea9b22" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.941414 595 raft_consensus.cc:3032] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.941447 595 raft_consensus.cc:729] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.941567 595 consensus_queue.cc:260] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.941609 464 raft_consensus.cc:1261] T 4584b80c6d7e4feca28d7b321bca613f P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.941720 595 raft_consensus.cc:3037] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.941855 465 raft_consensus.cc:1261] T 99333b8e2108485cb56bee5d5fd22d25 P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.942034 328 raft_consensus.cc:2454] T 9d752855971a4015b1ebc62993ea9b22 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 2.
I20240426 09:51:04.942487 595 raft_consensus.cc:2454] T 9d752855971a4015b1ebc62993ea9b22 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 2.
I20240426 09:51:04.942660 2446 mvcc.cc:205] Tried to move back new op lower bound from 7021056265969147904 to 7021056265639075840. Current Snapshot: MvccSnapshot[applied={T|T < 7021056265969147904}]
I20240426 09:51:04.943173 32601 leader_election.cc:302] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, 4076eb0699724ee78c2f2f0d6354cf9b; no voters: f9655c9ff2e14edfbcd798d88bca0623
I20240426 09:51:04.943452 2432 raft_consensus.cc:2781] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Leader election won for term 3
I20240426 09:51:04.943506 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.943589 32663 raft_consensus.cc:482] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.943639 2432 raft_consensus.cc:686] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.943768 2432 consensus_queue.cc:237] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.944144 32598 leader_election.cc:302] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.944547 2432 raft_consensus.cc:2781] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.944648 2432 raft_consensus.cc:686] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.944743 2432 consensus_queue.cc:237] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.943652 32663 raft_consensus.cc:3037] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.946115 32663 raft_consensus.cc:504] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.946415 32663 leader_election.cc:288] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:04.947077 2153 consensus_queue.cc:1035] T 4584b80c6d7e4feca28d7b321bca613f P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.947419 2153 consensus_queue.cc:1035] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.947948 32665 raft_consensus.cc:1261] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.947887 32522 catalog_manager.cc:5484] T c4fbe777c8f94d80b3ae5df7bef3b724 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 2 to 3, leader changed from f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132) to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 3 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:04.948004 328 raft_consensus.cc:1261] T 679d7a8559df4e6fbae5e44dc3b2e227 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.948103 32522 catalog_manager.cc:5484] T 9d752855971a4015b1ebc62993ea9b22 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 1 to 2, leader changed from f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132) to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
I20240426 09:51:04.950477 2444 consensus_queue.cc:1035] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.950616 1867 consensus_queue.cc:1035] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.951615 328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.951719 328 raft_consensus.cc:3037] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.952682 328 raft_consensus.cc:2454] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
I20240426 09:51:04.953158 32601 leader_election.cc:302] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:51:04.953318 1725 raft_consensus.cc:2781] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Leader election won for term 3
I20240426 09:51:04.953415 1725 raft_consensus.cc:686] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.953537 1725 consensus_queue.cc:237] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.955227 32522 catalog_manager.cc:5484] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 2 to 3, leader changed from f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132) to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 3 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } }
I20240426 09:51:04.956441 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eae2c486afa483ab3ed754cc720ed8f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.956550 326 raft_consensus.cc:3037] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.957551 326 raft_consensus.cc:2454] T 2eae2c486afa483ab3ed754cc720ed8f P 07593a4e1c0242a1b52bb96338b63c8c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
I20240426 09:51:04.957742 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9771706a67b4a12b64cc43a991bbd27" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 8 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.957861 595 raft_consensus.cc:3032] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Stepping down as leader of term 2
I20240426 09:51:04.957904 595 raft_consensus.cc:729] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.957958 32601 leader_election.cc:302] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters: f9655c9ff2e14edfbcd798d88bca0623
I20240426 09:51:04.958026 595 consensus_queue.cc:260] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.958178 595 raft_consensus.cc:3037] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.958305 2432 raft_consensus.cc:2781] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Leader election won for term 3
I20240426 09:51:04.958380 2432 raft_consensus.cc:686] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.958495 2432 consensus_queue.cc:237] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.959125 595 raft_consensus.cc:2454] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
I20240426 09:51:04.959852 325 raft_consensus.cc:1261] T f34e4326a7cd44669b9a58324d883911 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.960448 2444 consensus_queue.cc:1035] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.960680 32517 catalog_manager.cc:5484] T 2eae2c486afa483ab3ed754cc720ed8f P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 2 to 3, leader changed from f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132) to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 3 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } }
I20240426 09:51:04.960763 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.960888 32663 raft_consensus.cc:482] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.960911 32665 raft_consensus.cc:1261] T f34e4326a7cd44669b9a58324d883911 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Refusing update from remote peer f9655c9ff2e14edfbcd798d88bca0623: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.960940 32663 raft_consensus.cc:3037] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.961843 32663 raft_consensus.cc:504] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.962189 32663 leader_election.cc:288] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:04.962668 462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
W20240426 09:51:04.964498 556 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:58568: Illegal state: replica f9655c9ff2e14edfbcd798d88bca0623 is not leader of this config: current role FOLLOWER
I20240426 09:51:04.966140 2519 consensus_queue.cc:1035] T f34e4326a7cd44669b9a58324d883911 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Connected to new peer: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.966351 32665 raft_consensus.cc:1226] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Rejecting Update request from peer f9655c9ff2e14edfbcd798d88bca0623 for earlier term 2. Current term is 3. Ops: []
I20240426 09:51:04.967087 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99333b8e2108485cb56bee5d5fd22d25" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.967185 595 raft_consensus.cc:3032] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Stepping down as leader of term 2
I20240426 09:51:04.967222 595 raft_consensus.cc:729] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.967232 2512 consensus_queue.cc:1046] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.967393 595 consensus_queue.cc:260] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.967501 595 raft_consensus.cc:3037] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.968453 595 raft_consensus.cc:2454] T 99333b8e2108485cb56bee5d5fd22d25 P f9655c9ff2e14edfbcd798d88bca0623 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
I20240426 09:51:04.968637 2431 mvcc.cc:205] Tried to move back new op lower bound from 7021056266008711168 to 7021056265587019776. Current Snapshot: MvccSnapshot[applied={T|T < 7021056266008711168}]
I20240426 09:51:04.968649 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3abebac937b44862950b2f1957cca770"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.968740 32663 raft_consensus.cc:482] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.968791 32663 raft_consensus.cc:3037] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.969692 32663 raft_consensus.cc:504] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.970052 32663 leader_election.cc:288] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 2 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
W20240426 09:51:04.971390 556 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:58598: Illegal state: replica f9655c9ff2e14edfbcd798d88bca0623 is not leader of this config: current role FOLLOWER
I20240426 09:51:04.972087 327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abebac937b44862950b2f1957cca770" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "07593a4e1c0242a1b52bb96338b63c8c"
I20240426 09:51:04.972193 327 raft_consensus.cc:3032] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Stepping down as leader of term 1
I20240426 09:51:04.972234 327 raft_consensus.cc:729] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07593a4e1c0242a1b52bb96338b63c8c, State: Running, Role: LEADER
I20240426 09:51:04.972348 327 consensus_queue.cc:260] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.972435 32598 leader_election.cc:302] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters: 4076eb0699724ee78c2f2f0d6354cf9b
I20240426 09:51:04.972496 327 raft_consensus.cc:3037] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.972690 595 raft_consensus.cc:1261] T e9771706a67b4a12b64cc43a991bbd27 P f9655c9ff2e14edfbcd798d88bca0623 [term 3 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240426 09:51:04.972864 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.972944 32663 raft_consensus.cc:482] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.972986 32663 raft_consensus.cc:3037] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.973045 2432 raft_consensus.cc:2781] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Leader election won for term 3
I20240426 09:51:04.973122 2432 raft_consensus.cc:686] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.973229 2432 consensus_queue.cc:237] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } }
I20240426 09:51:04.973414 327 raft_consensus.cc:2454] T 3abebac937b44862950b2f1957cca770 P 07593a4e1c0242a1b52bb96338b63c8c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 2.
I20240426 09:51:04.973819 32663 raft_consensus.cc:504] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.974117 32663 leader_election.cc:288] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
W20240426 09:51:04.974494 2446 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:58572: Illegal state: Replica 07593a4e1c0242a1b52bb96338b63c8c is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } }
I20240426 09:51:04.974792 465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3abebac937b44862950b2f1957cca770" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.974900 465 raft_consensus.cc:3037] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.975075 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "713fa4d62643470285727c0d0092676f"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.975164 32663 raft_consensus.cc:482] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.975207 32663 raft_consensus.cc:3037] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.975910 465 raft_consensus.cc:2454] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 2.
I20240426 09:51:04.975910 32663 raft_consensus.cc:504] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.976084 398 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_5 already exists
I20240426 09:51:04.976220 32663 leader_election.cc:288] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Requested vote from peers f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661), 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223)
I20240426 09:51:04.976243 398 test_util.cc:268] Using random seed: -677824656
I20240426 09:51:04.976406 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.976500 595 raft_consensus.cc:3032] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Stepping down as leader of term 2
I20240426 09:51:04.976532 595 raft_consensus.cc:729] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.976653 595 consensus_queue.cc:260] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.976789 595 raft_consensus.cc:3037] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.977057 328 raft_consensus.cc:1261] T e9771706a67b4a12b64cc43a991bbd27 P 07593a4e1c0242a1b52bb96338b63c8c [term 3 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 3 index: 10. (index mismatch)
I20240426 09:51:04.977062 1725 consensus_queue.cc:1035] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240426 09:51:04.977383 32601 leader_election.cc:302] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07593a4e1c0242a1b52bb96338b63c8c, 342778fb25b446c68cdcd20f98bcd8b2; no voters:
I20240426 09:51:04.977710 595 raft_consensus.cc:2396] T b9315a8a86f942bdbcb12f96b1c4acec P f9655c9ff2e14edfbcd798d88bca0623 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 342778fb25b446c68cdcd20f98bcd8b2 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240426 09:51:04.977735 1725 raft_consensus.cc:2781] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 09:51:04.977797 1725 raft_consensus.cc:686] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.977901 1725 consensus_queue.cc:237] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.977977 32664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "451f3108a6fb494da331dd6a6ec3e290"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.978075 32664 raft_consensus.cc:465] T 451f3108a6fb494da331dd6a6ec3e290 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.979212 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9315a8a86f942bdbcb12f96b1c4acec" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.979272 463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713fa4d62643470285727c0d0092676f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.979322 464 raft_consensus.cc:3037] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.979339 463 raft_consensus.cc:3037] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.979986 32664 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "679d7a8559df4e6fbae5e44dc3b2e227"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.980082 32664 raft_consensus.cc:482] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.980130 32664 raft_consensus.cc:3037] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.980142 463 raft_consensus.cc:2454] T 713fa4d62643470285727c0d0092676f P 4076eb0699724ee78c2f2f0d6354cf9b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
I20240426 09:51:04.980214 464 raft_consensus.cc:2454] T b9315a8a86f942bdbcb12f96b1c4acec P 4076eb0699724ee78c2f2f0d6354cf9b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
W20240426 09:51:04.980901 32756 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:58562: Illegal state: replica 07593a4e1c0242a1b52bb96338b63c8c is not leader of this config: current role FOLLOWER
I20240426 09:51:04.980998 32664 raft_consensus.cc:504] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.981289 32664 leader_election.cc:288] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Requested vote from peers 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:04.981616 596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679d7a8559df4e6fbae5e44dc3b2e227" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.981632 2481 mvcc.cc:205] Tried to move back new op lower bound from 7021056266008711168 to 7021056265587019776. Current Snapshot: MvccSnapshot[applied={T|T < 7021056266008711168 or (T in {7021056266008711168})}]
I20240426 09:51:04.981709 596 raft_consensus.cc:3032] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Stepping down as leader of term 2
I20240426 09:51:04.981743 596 raft_consensus.cc:729] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.981868 596 consensus_queue.cc:260] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.981977 596 raft_consensus.cc:3037] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.982270 595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "713fa4d62643470285727c0d0092676f" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f9655c9ff2e14edfbcd798d88bca0623"
I20240426 09:51:04.982355 595 raft_consensus.cc:3032] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Stepping down as leader of term 2
I20240426 09:51:04.982390 595 raft_consensus.cc:729] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f9655c9ff2e14edfbcd798d88bca0623, State: Running, Role: LEADER
I20240426 09:51:04.982504 595 consensus_queue.cc:260] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.982651 595 raft_consensus.cc:3037] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 2 FOLLOWER]: Advancing to term 3
I20240426 09:51:04.982730 596 raft_consensus.cc:2454] T 679d7a8559df4e6fbae5e44dc3b2e227 P f9655c9ff2e14edfbcd798d88bca0623 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
I20240426 09:51:04.982956 32601 leader_election.cc:302] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, 4076eb0699724ee78c2f2f0d6354cf9b; no voters:
I20240426 09:51:04.983093 32601 leader_election.cc:302] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, 4076eb0699724ee78c2f2f0d6354cf9b; no voters: f9655c9ff2e14edfbcd798d88bca0623
I20240426 09:51:04.983477 32598 leader_election.cc:302] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 342778fb25b446c68cdcd20f98bcd8b2, f9655c9ff2e14edfbcd798d88bca0623; no voters:
I20240426 09:51:04.983558 595 raft_consensus.cc:2454] T 713fa4d62643470285727c0d0092676f P f9655c9ff2e14edfbcd798d88bca0623 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 3.
I20240426 09:51:04.984539 2542 raft_consensus.cc:2781] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Leader election won for term 3
I20240426 09:51:04.984570 464 raft_consensus.cc:1261] T 3abebac937b44862950b2f1957cca770 P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Refusing update from remote peer 342778fb25b446c68cdcd20f98bcd8b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240426 09:51:04.984462 32517 catalog_manager.cc:5484] T 99333b8e2108485cb56bee5d5fd22d25 P 342778fb25b446c68cdcd20f98bcd8b2 reported cstate change: term changed from 2 to 3, leader changed from f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132) to 342778fb25b446c68cdcd20f98bcd8b2 (127.0.99.129). New cstate: current_term: 3 leader_uuid: "342778fb25b446c68cdcd20f98bcd8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } }
I20240426 09:51:04.984690 2542 raft_consensus.cc:686] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.984839 2542 consensus_queue.cc:237] T 713fa4d62643470285727c0d0092676f P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } }
I20240426 09:51:04.985473 2542 raft_consensus.cc:2781] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Leader election won for term 3
I20240426 09:51:04.985579 2542 raft_consensus.cc:686] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.986939 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "76ada11863ac4cce802de521b842be2d"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.987349 32663 raft_consensus.cc:465] T 76ada11863ac4cce802de521b842be2d P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240426 09:51:04.987025 2545 consensus_queue.cc:1035] T e9771706a67b4a12b64cc43a991bbd27 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240426 09:51:04.988898 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "739229b67c4d49e7970c6cd9eb47690e"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.988988 32663 raft_consensus.cc:482] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.989039 32663 raft_consensus.cc:3037] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.989892 32663 raft_consensus.cc:504] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } } peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } }
I20240426 09:51:04.990074 2432 raft_consensus.cc:2781] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 FOLLOWER]: Leader election won for term 3
I20240426 09:51:04.990159 32663 leader_election.cc:288] T 739229b67c4d49e7970c6cd9eb47690e P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 2 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), 07593a4e1c0242a1b52bb96338b63c8c (127.0.99.130:37271)
I20240426 09:51:04.990176 2432 raft_consensus.cc:686] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [term 3 LEADER]: Becoming Leader. State: Replica: 342778fb25b446c68cdcd20f98bcd8b2, State: Running, Role: LEADER
I20240426 09:51:04.990289 2432 consensus_queue.cc:237] T 679d7a8559df4e6fbae5e44dc3b2e227 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07593a4e1c0242a1b52bb96338b63c8c" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37271 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.990360 2545 consensus_queue.cc:1035] T 3abebac937b44862950b2f1957cca770 P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240426 09:51:04.990914 32663 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3c8353b8e1494e33a069eb6921b81ba5"
dest_uuid: "342778fb25b446c68cdcd20f98bcd8b2"
from {username='slave'} at 127.0.0.1:33888
I20240426 09:51:04.991003 32663 raft_consensus.cc:482] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 09:51:04.991045 32663 raft_consensus.cc:3037] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.991712 464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "739229b67c4d49e7970c6cd9eb47690e" candidate_uuid: "342778fb25b446c68cdcd20f98bcd8b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "4076eb0699724ee78c2f2f0d6354cf9b"
I20240426 09:51:04.991747 32663 raft_consensus.cc:504] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4076eb0699724ee78c2f2f0d6354cf9b" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46223 } } peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132" port: 40661 } } peers { permanent_uuid: "342778fb25b446c68cdcd20f98bcd8b2" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36999 } }
I20240426 09:51:04.991811 464 raft_consensus.cc:3037] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [term 1 FOLLOWER]: Advancing to term 2
I20240426 09:51:04.992564 32663 leader_election.cc:288] T 3c8353b8e1494e33a069eb6921b81ba5 P 342778fb25b446c68cdcd20f98bcd8b2 [CANDIDATE]: Term 2 election: Requested vote from peers 4076eb0699724ee78c2f2f0d6354cf9b (127.0.99.131:46223), f9655c9ff2e14edfbcd798d88bca0623 (127.0.99.132:40661)
I20240426 09:51:04.992712 464 raft_consensus.cc:2454] T 739229b67c4d49e7970c6cd9eb47690e P 4076eb0699724ee78c2f2f0d6354cf9b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 342778fb25b446c68cdcd20f98bcd8b2 in term 2.
I20240426 09:51:04.992749 2542 consensus_queue.cc:237] T b9315a8a86f942bdbcb12f96b1c4acec P 342778fb25b446c68cdcd20f98bcd8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9655c9ff2e14edfbcd798d88bca0623" member_type: VOTER last_known_addr { host: "127.0.99.132"